PWAs are back

02. March 2024 · Categories: Apple, Politics

Apple has now confirmed that they will continue supporting Progressive Web Apps in the EU, iOS will serve them using Webkit. Over at gamesfray, Florian Müller explains why he believes this to be illegal. I believe this interpretation is wrong, that Apple overreacted, and that PWA never fell under the interoperability clauses of the Digital Markets Act. The important parts are

  • article 5, part 7:

    The gatekeeper shall not require end users to use, or business users to use, to offer, or to interoperate with, an identification service, a web browser engine or a payment service, or technical services that support the provision of payment services, such as payment systems for in-app purchases, of that gatekeeper in the context of services provided by the business users using that gatekeeper’s core platform services.

  • article 6, part 7:

    The gatekeeper shall allow providers of services and providers of hardware, free of charge, effective interoperability with, and access for the purposes of interoperability to, the same hardware and software features accessed or controlled via the operating system or virtual assistant listed in the designation decision pursuant to Article 3(9) as are available to services or hardware provided by the gatekeeper. Furthermore, the gatekeeper shall allow business users and alternative providers of services provided together with, or in support of, core platform services, free of charge, effective interoperability with, and access for the purposes of interoperability to, the same operating system, hardware or software features, regardless of whether those features are part of the operating system, as are available to, or used by, that gatekeeper when providing such services.

    The gatekeeper shall not be prevented from taking strictly necessary and proportionate measures to ensure that interoperability does not compromise the integrity of the operating system, virtual assistant, hardware or software features provided by the gatekeeper, provided that such measures are duly justified by the gatekeeper.

PWAs are in my opinion not covered. In 5(7), only business users using the core platform service (iOS) get the right to chose their web browser engine. This means iOS apps, not web pages1. And 6(7) only demands that you get access to the same features that Apple uses, it does not include the right to replace the underlying implementation. So it only means that rival web browsers will be able to install PWAs, not replace the underlying browser engine for PWAs. The Safari App does not run PWAs, this is done by iOS with the help of Webkit, this is thus not an ability rival web browsers could demand access to.

  1. Were we to regard the browser as a core platform service, then a literal reading could require the browsers to allow web sites to select an engine of their choice. This is technically absurd, and legally dubious, as this is obviously aimed at forced bundling, as clearly laid out in note 43. And I doubt the DMA wants to force end users to tolerate crypto mining engines destroying their batteries. ⏎

Apple’s Core Technology Fee is Clearly Illegal

26. January 2024 · Categories: Apple, Politics

Apple has now published their proposal to comply with the Digital Markets Act. Apple now allows you to avoid the App Store while charging €0.50 annually for all app installs above a million. This is in clear violation of Article 6, §7 of the DMA, which says:

The gatekeeper shall allow providers of services and providers of hardware, free of charge, effective interoperability with, and access for the purposes of interoperability to, the same hardware and software features accessed or controlled via the operating system or virtual assistant listed in the designation decision pursuant to Article 3(9) as are available to services or hardware provided by the gatekeeper. Furthermore, the gatekeeper shall allow business users and alternative providers of services provided together with, or in support of, core platform services, free of charge, effective interoperability with, and access for the purposes of interoperability to, the same operating system, hardware or software features, regardless of whether those features are part of the operating system, as are available to, or used by, that gatekeeper when providing such services.

The gatekeeper shall not be prevented from taking strictly necessary and proportionate measures to ensure that interoperability does not compromise the integrity of the operating system, virtual assistant, hardware or software features provided by the gatekeeper, provided that such measures are duly justified by the gatekeeper.

It is important to understand that the App Store is separate from iOS, according to the DMA. Which means that Apple has to provide third parties access to iOS on the same terms as it uses for the App Store and its other built in apps. It cannot charge developers for that access, but it could charge users. Maybe 0.50€ per app per year, with the first 20 free. But then it would need to count the built in apps against that, and removing a built in app would free a slot for an external app, so that users can substitute Google Maps for Apple Maps. And they need to charge the same fee independently of where the app came from, App Store apps would incur the same costs as apps from other sources. This is clearly undesirable for Apple, to create an incentive to throw out built in apps to save money, and to no longer sample apps from the App Store. It also economically advantages large cooperations, and incentives them to create a single mega app, in order to save on the fees. This is not good from a policy point of view.

The Enduring Myth of Climate Destroying Video Streaming

31. December 2023 · Categories: Hardware, Politics

I recently came across a paper on the energy costs of video streaming, written by the Borderstep Institute in German. It repeats the myth of insanely high streaming energy costs. That seems to have started with a fatally flawed analysis by the Shift Project, now reasonably refuted by the IEA.

The Borderstep paper takes the average data transfer volume and energy consumption of data centers, as well as that for the broadband network, calculates an average efficiency of 30 kBit/Ws for broadband, and 70 kBit/Ws for the data centers, and uses that to extrapolate 1000W power consumption for streaming a 25 MBit/s 4K stream. This result is wrong, because it neglects the actual engineering:

  • Streaming uses very little processing power on the server side

    All popular videos are stored already fully encoded on the servers, in multiple bit rates, to improve responsiveness. Even a typical home NAS, drawing 40W power, can stream 40 4K streams in parallel over a 1 Gbit/s connection. In the data centers, we see units drawing 1000W serving 20 Gbit/s. Even with a generous allowance for overheads and reserve capacity I’d be surprised if a 4K stream draws more than 5W, for an efficiency of 5 Mbit/Ws. Given that the typical HDD delivers 50 to 100 Mbit/Ws, this feels reasonable.

  • Streaming is mostly via CDNs, and the backbone is now fiber

    With a Content Delivery Network, the data will only have to travel a few hops and at most a few hundred kms, and this can be done efficiently with fiber. DWDM type connections manage 160km with 1 Gbit/s with the transceiver drawing 4W per side, that is 125 Mbit/Ws. Assuming that the switch draws the same power, that we need at most 5 hops, and adding a 5x safety margin, we end up at 2.5 MBit/Ws. This is 10W for the backbone, likely even lower.

  • The last mile for broadband has basically a fixed power consumption

    While in the backbone and data centers you aggregate demand, and so can ensure a pretty high load and efficiency, the last mile is essentially always on. So the energy consumption stays the same, no matter whether you use it sparingly for surfing, or intensely for streaming. You draw 10W if you have fiber, 20W if you have DSL. Your WiFi router at home will also draw a fixed 5 to 10 Watts.

A 4K stream with fiber to the home, and an efficient router will draw up to 5W + 10W + 10W + 5W = 30W, a low bit rate stream over DSL with an old router will be 0.5W + 1W + 20W + 10W = 31.5W. This is similar to the 18W for data transmission the IEA assumes, without taking your home wifi into account. It also shows that with current technology 4K is roughly the tipping point where the backbone starts to consume more power than the last mile, and we need to start monitoring our consumption. Still, this is 300 GB/day.

If you are worried about the climate impact of streaming, look at the TVs instead. A 65″ HD display draws 200W, six times the power needed to move the data.

Replacing a Synology NAS

15. November 2023 · Categories: Hardware

I recently decided to replace my old Synology NAS, a DS416, with a more powerful version, a DS423+. This turned out to take much longer than expected, mainly because I also wanted to take the opportunity to also start using btrfs as a more robust filesystem, and so needed to migrate my data as well.

For the migration, Synology recommends using their backup solution, Hyper Backup, but their steps are not efficient, if you can live with a service interruption of a few days. Their suggestion is to run a backup of the old system, restore to a new system, stop using the old system, repeat transfer with incremental backup, switch over to the new system, and start service again. That works fine with powerful hardware, and with enough space for all the backups, and it makes sure that all permissions are copied correctly.

This description leads you astray if your old NAS is slow, you lack space, or you can tolerate a longer interruption. With a slow NAS, encryption can become a bottleneck, and a backup of an encrypted folder to an encrypted target over an encrypted connection meant that backup speeds were below 15 MB/s. Instead do the following:

  • Move the disks directly to your new NAS. Use Synology Assistant on your computer to verify that you can install DSM without loss.

  • Prepare your old NAS for backup duty: put in empty disks, create a redundant storage pool on them to receive your data, add a volume with shared folder, install Hyper Backup Vault. If you needed to grab some SMR disks, trim them with fstrim via SSH.

  • Backup the data to the old NAS, including all package info. I used Hyper Backup for this, with client-side encryption for the backup so that there is only one encryption step, and it is done by the powerful new NAS. Still I had only around 45 MB/s backup and 40 MB/s restore speeds with my 5400rpm HDDs.

  • Delete the volumes on the new NAS, and create btrfs volumes instead. Make a note of the packages that are uninstalled to put them back later. Recreate the shared folders before you run the restore, so that you have control over which folders should have integrity checks enabled.

  • Restore the backup to the new NAS.

Since Hyper Backup is quite slow as noted, you might want to use direct copies instead. This can give problems with permissions, and means a slightly higher risk of data loss. On my old NAS, where I wanted to keep encryption, it was even slower at 30 MB/s as doing encryption twice (transfer and rest) slowed it down.

Using SMR Disks

08. November 2023 · Categories: Hardware

Shingled magnetic recording is a hard disk recording method where multiple tracks are written in a slightly overlapping manner1. Because the write head is larger than the read head, mainly because it needs to generate a strong enough magnetic field to affect the disk, you can increase density by partially overwriting the data you are recording. If you are looking at it by height, you would get with a 50% overlap, and four tracks:

Action A B C D D2
Write A A A ? ? ?
Write B A B B ? ?
Write C A B C C ?
Write D A B C D D

You see that you can increase density nicely, but in order to update any track but D you would also need to rewrite the tracks below them to prevent data loss. Assuming that you group your tracks into groups of 8, you would roughly need 10 half tracks in height (one half track is needed for synchronization info), instead of 16 with conventional recording, but worst case your write speed is reduced by a factor of 8 when updating track A.

This is awful performance for random access data, but can be fast enough with only large files. The OS can then tell the drive which sectors have been deallocated, and the disk no longer needs to restore those tracks. This is done with a TRIM command. In addition, you want a small percentage of the disk set aside to use conventional recording, so that you can update your file meta data quickly. This seems to be rarely implemented.

Under Linux, you can use hdparm -I /dev/sda to get detailed HDD info. fstrim can be used to trim unused space on a partition, if the filesystem does not do this automatically. This is especially useful for a freshly reformatted disk to ensure the disk firmware treats the data as unused.

  1. There is a more technical overview at zonedstorage.io ⏎

M3 Pricing Matrix

31. October 2023 · Categories: Apple

The pricing matrix for the M3 has become quite complicated:

Processor 8GB 16GB 18GB 24GB 36GB 48GB 64GB 96GB 128GB
M3- -200 0 200
M3 0 200 400
M3 Pro 400 800
M3 Pro+ 600 1000
M3 Max 1400 2200
M3 Max+ 1900 2100 2900

This looks like an attempt to steer people to manage Max and Max+ demand, especially the lack of a 64GB Max variant.

Tidbits:

  • the improved graphics cache for the Pro/Max models provides a surprising amount of extra graphics power. This moves them much closer to top class graphics performance.

  • memory bandwidth for M3 Pro is only 150GB/s, M3 Max 300 GB/s and M3 Max+ 400GB/s

  • the neural engine did not get any more cores, contrary to the 32 cores now on the A17 Pro.

  • while SSD prices have halved in the last year, Apple has kept their prices the same.

  • memory pricing remains the same. Cost for extra memory up to 36GB remains high, and double the price for later increments. Mobile DDR5 is $120 per 16GB, DDR5X maybe $150.

Update 08.11. As John Gruber has pointed out, the complications are basically a function of using 2, 3 or 4 memory chips with the processor. He also would have liked a 48GB M3 Pro model, something that would be the correct mobile sweet spot. After all an M3 Pro can already draw 50W under full load, and would become quite warm under continuous load. That 6+6 configuration looks quite sweet to keep heat in check.

I suspect it does not exist because it would clearly show the kinks in Apples memory pricing, a lamentable victory of marketing over user needs.

DRAM Power Consumption

29. October 2023 · Categories: General

I was surprised to be reminded how much power RAM can consume in a laptop or NAS. NAS typically use slightly older technology, for example the Sinology NAS can be expanded with dual rank DDR4 laptop modules. I found a Micron technical note giving a deep dive. From page 24 there is an example for a 4GB configuration, where they calculate 1.6W typical draw, of which 350mW is background power. The other state is self refresh, which on a 1GB module is 25mA + 5mA @1.2V (Page 324). This gives the following estimates for DDR4 power consumption, scaled to 16GB:

State Power
Used 6.4W
Unused 1.4W
Standby 0.6W

These are rough estimates, given that multiple modules on the same bus add additional parasitic draws.

As such, you should aim for 50% utilization under load. Less, and you pay quite a bit in power for unused capacity, more, and your risk of needing paging increases, which will consume more power than unused RAM would, and drastically reduces performance.

VMWare Fusion Display Issues

18. September 2023 · Categories: Software

VMWare Fusion allows you to run Windows on a Mac, and one of the issues with it is that both deal with high resolution text scaling differently. Windows uses ClearType to adjust the glyphs to look best on the physical display. This gives excellent results on 1x scaling with 110 dpi or less, but only if the output is not scaled afterwards. MacOS on the other hand no longer does subpixel trickery and only does standard antialiasing. This approach provides worse results at 1x, but it preserves more information for a scaled display, and looks better with scaling.

When your external display is 1x, you should use ClearType and 1x on Windows. Make sure that you center a full screen window, and not scale it automatically. Automatic scaling has an unfortunate bug where the display dimensions are somehow off by a pixel, and so stretches the pixel a bit, making text look terrible. Pretty much the same for 2x, only that you activate retina resolution for your VM, and do 200% text scaling in Windows.

If you have something in-between, like 140dpi or 160dpi, you have two options:

  • dedicate your external monitor to Windows, run it at a native resolution in macOS, and use 125% or 150% display scaling on Windows. This gives nice text, but fixes the monitor to be used for Windows only

  • use a scaled resolution in macOS, activate retina resolution in VMWare, run Windows with 200% scaling, and deactivate ClearType in Windows. This gives better letters for scaling, and it gives reasonable text in Windows. The text is worse than the alternative above, but it makes it possible use macOS with a similar text size in parallel.

Plug-In Cars are the future

31. August 2023 · Categories: Politics

One interesting article on the Distribution of car distances travelled shows that a plug in hybrid with 75km electrical range would be able to cover 75% to 80% of all driving using the battery alone, and it did not include the ability to recharge your car at work.

This makes them a really good policy choice to reduce carbon emissions. The worry with e-cars is that they will all come equipped with 80kWh batteries just to allow the rare road trip, a battery that even with just 1000 cycles is good for 400000km, and will likely deteriorate more from aging than from actual use. Instead a 75km plugin would just need a 12kWh battery, much closer to the actual live span of a car, and also much lighter, 100kg instead of over 400kg.

Currently there is not a singly electric car on the market that is affordable, and able to cover longer distances. A 25000€ petrol car can easily travel 800km in 6.5 hours, while starting with just 100km range remaining, but electric cars with sufficient charge speed for this are only available with huge batteries, raising the cost into the 80000€ plus range, and requiring you to travel at more than 160 kph to reach that average speed.

The plugin will for 1/5th the battery resources enable a 80% reduction in fuel consumption, reduce city pollution by having enough range to run there fully electric, produce half the road damage, be less dangerous to others in a crash, and would not require huge upfront costs for buying a car. As such it is a much better policy choice than forcing everyone onto super heavy electric cars.

The alternative would be for sufficient cooling on the batteries to allow a 5 minute fill up. Then people would be willing to use a 250km range car, knowing that extra range is a fast top up, and not a half hour wait. A few such experiments are now conducted, and I sure hope this becomes standard soon. It would also require a massive extension of fast charging stations on highways, half of which would only be used during a handful of peak holiday travel days. I suspect the cars are ready much earlier than the charging infrastructure.

Does Strasbourg really want to forbid AirPods?

25. July 2023 · Categories: Politics

The EU is currently considering a new battery directive1, and as it stands, it will effectively ban AirPods, and probably even smartwatches.

The problem is article 11, which basically forces everything using batteries weighting less than 5 kg to use user replaceable batteries, with only one exception when “specifically designed to operate primarily in an environment that is regularly subject to splashing water, water streams or water immersion, and that are intended to be washable or rinseable”. That makes it applicable for earbuds, smart watches, phones, and tablets. All of these will become seriously worse products because of this misguided law.

To understand why, a user replaceable battery needs to be more robust. It will have a plastic housing instead of directly fitting the battery pack, the housing will have significantly less flexibility to fit a battery tightly, and you need to properly secure the pack. User replaceable also means that you no longer can use glue to seal your device water tight. Instead you will use some kind of rubber seal, as they were used in tough compact cameras, like the Olympus TG-6. From experience, you need to be careful. The surfaces for the seal must be clean for the seal to be any good, especially there must never be any sand on them. Then you need to fasten the seals and batteries. You can either use a cap as in cameras, which use up a lot of space, or you use screws, which are a potential ingress point for water, and you also run the risk that users will fasten them wrongly. You really want a torque sensor to tighten them properly. In short, a user replaceable battery increases weight, and volume, and makes environmental sealing heavier. And for batteries that last at least as long as the product, they add avoidable waste.

Looking at the Fairphone and the iPhone 14, you see the impact a replaceable battery can have: 225g instead of 172g, IP54 rated instead of IP68. Apart from a few idealist, people just prefer the much better phone, and with Apple offering battery replacement for €119, this is a tradeoff many are willing to make.

For smartwatches, this is a disaster. I want my watch on during a swim, so it needs really good protection, and I want it to be smallish2. Design goals that currently cannot be achieved with replaceable batteries. But a watch is primarily used outside the water, even if swimming is critical use case, so the exception in the law does not take.

And for AirPods, they are worn inside our ears, so from an ergonomic perspective they need to be as light as possible. One AirPod Pro is just 5.3g, so even adding 1g to support a replaceable battery is bad. Apple uses a 25mAh battery, which I estimate to weight around 500mg, comparing them with a 210mAh, 3.2g CR2032. And the environment calls for good sealing, as you do not want your sweat to destroy them.

These are real design tradeoffs, and the law needs to recognize this. This means that it should be possible to have a battery replacement service as an alternative, when other design goals would be compromised. Difficult to put this into legal requirements, but let us try.

Amendment to Article 11:

  1. Devices are conditionally exempt from the user changeable requirement when one or more important design goals recognized in this article would be negatively impacted

  2. Exempt devices need to conform to all of the points below

    1. be designed that their batteries allow at least 1500 charge cycles, or that they last for at least 3 years for at least 95% of users before needing replacement.

    2. provide a battery replacement service that costs at most 50€ in 2023 euros more than a typical battery of roughly the same capacity, volume and endurance.

    3. once the battery replacement service is stopped, the necessary documentation must be provided free of charge for third parties to provide this service instead. Any necessary patents and other rights need to be licensed on a FRAND basis. This obligation includes tools, processes and replacement parts.

    4. use at least 20% of their volume or weight for batteries, or weight less than 100g.

  3. Important design goals are for

    1. Asset trackers:
      1. Robustness against manipulation

      2. Environmental sealing

      3. Weight

    2. Devices that are designed to be worn on the body, taken with you all day, or to be used handheld for at least 15 minutes continuously:

      1. Weight

      2. Size

      3. Environmental sealing

      4. Robustness against rough handling

    3. Devices intended to be portable, and be used at multiple locations. These can be exempt if the smallest dimension of their rectangular bounding box is less than 20mm.

      1. Weight

      2. Size

    4. Devices intended to be used regularly in a harsh environment. These devices do not need to meet the volume/weight threshold from 2.4.

      1. Environmental sealing

      2. Robustness against rough handling

  4. This conditional exemption is valid until two years after a comparable device with a user replaceable battery is available on the market that provides for those design goals at least as well, and with a volume and weight penalty of at most 5% of the entire device, with comparable life time hardware costs.

  1. The law text, as adapted in first reading by the parliament, is found here ⏎

  2. The Apple Watch 8 45mm is 39g, with a 310mAh battery. That battery would be an estimated 5g to 6g. ⏎
« Older articles