Zigbee vs. Matter over Thread:Understanding IoT Protocol Performance in Practice
arxiv.org105 points by teleforce 2 days ago
105 points by teleforce 2 days ago
I’m in the process of migrating from Zigbee (Z2M) to Matter over Thread (Apple HomePods + Matter Server).
My Zigbee network was a mishmash of different vendors all of which had their own failure rates and quirks, Aqara having by far the highest failure rate and complexity.
My new Ikea (Thread) based setup has had absolutely zero issues so far and fingers crossed it doesn’t start suffering as I scale it out. That said, it is noticeably slower than Zigbee, but we’re talking 50ms > 250ms. Which is more than acceptable.
What's to stop Ikea from pushing an OTA and crippling your whole setup 2 years from now?
You don't need to update your devices, and they don't need to be connected to any cloud. That's the whole point of these local communication protocols, after all.
Sorry to pop your bubble, but the moment you add any IP connectivity it will be eventually used to mandate internet access. The devices you have now might not yet demand it, but you will have to replace them at some point or extend your setup in the future. We've seen this play out so many times now, it's baffling people still think this time will be different.
There's a saying: If you set up a trough, the pigs will come.
I run 100-device zigbee network with zigbee2mqtt. Philips, Bosch, Sonoff and some others. No internet access.
Zigbee devices are slow and low power, they cannot access internet directly.
Not directly, no; they do it via the hub
Zigbee: You can buy vendor independant bridge / antenna and vendor independant automation server (Home Assistant being the most popular).
I guess a bridge vendor could pull the rug on the antenna ("your Sonoff zigbee antenna is now only compatible with Sonoff zigbee product")(to be clear: they've never done that), but the format is open enough that there's ton of alternative suppliers and open source bridge (the aforementioned zigbee2mqtt).
In any case, I've got a good amount of both Ikea tradfri and Philips Hur bulb and they have never contacted an Ikea or Philips hardware or software of their life.
For IPv6 networks you're in luck. The hubs are not proxies so the IPv6 enabled IoT device in an IPv6 network can still be blocked at firewall level from accessing the internet. Less that the separation Zigbee provides but better than nothing.
For IPv4 networks you're out of luck. The internal mesh traffic is still IPv6 to the hub and anything crossing into the IPv4 network is done through the hub's IP. No more blocking available unless it's a feature of the hub. I'm not aware of any hub having this kind of FW capability.
Firewall blocking is not the point. The point is, that over time through enshitiffication the new device will require call home to unlock. Even when it does not need internet to work at all. But through this the vendor will get a foothold.
Sure you can block it, but then you have a pricey paperweight.
This is not unique to Thread/Matter, they can pull the rug on ZigBee as well, ofc.
> through enshitiffication the new device will require call home to unlock
I thought Matter certification demands that the device work without cloud connectivity, it's part of the spec. Being able to use any hub and have guaranteed basic functionality without the cloud is the big selling point.
> This is not unique to Thread/Matter, they can pull the rug on ZigBee as well, ofc.
Exactly. Enshittification can't be prevented with technical means, a manufacturer looking to rentseek can always do something shady to force your hand no matter the technology. These are by definition smart and connected devices, there are plenty of avenues for abuse. Philips tried to push an app update that required you to log in to an account to manage your Zigbee Hue hub and devices. They'll try again.
Thread/Matter are on paper mostly good for the consumers but come with some potential drawbacks. No smart home technology eliminates these risks, they only balance different advantages and drawbacks.
> but then you have a pricey paperweight.
The device must work offline. You might lose some features though, usually the ones that are only supported via the proprietary app.
The assumption of enshitification it that is receives updates. What you and a few others aren't understanding is that these don't need updates. These aren't Alexa or Google devices. They just get setup and passively work
Now, will they eventually build devices that don't work like that? Maybe.
The enshitification I mean, applies to the new models. Once the current device dies, you need to replace it.
And in the future, the offline devices might be no longer available, because they stopped being produced. And only new shittier ones are for sale.
Oh you're talking about a pretend device that this person with replace due to the pretend problem with their current device.
So what's your solution to this problem?
A lot of companies have invested a lot into matter and thread. I’d be surprised if companies started requiring a call home to work. It doesn’t make sense to need that, since a thread network is, in a sense, offline by nature. Only the hub has access to the internet and it doesn’t require access to work, so having a device require a call home wouldn’t really make sense.
I get that enshittification is a real thing but I’d be surprised if it affected smart home devices in the way you are describing.
i dont really understand the point. am i never going to buy any tech ever again, because most things tend toward enshitification?
That they would never see me buying again from them. That I can stop OTA. That matter is interoperable and I can run multiple gateways before cutting them out.
Most IoT stuff I own had a good track record. The only stuff that screwed me over was the logitech harmony smart remote.
But I usually pick the more open ecosystems and vendors. E.g.heating is on shelly.
If we start from GP premises where they were multi-vendor on ZigBee and now (allegedly) are living a much better mono-vendor life with Matter, you cannot play the multi-vendors card. I'm on ZigBee multi-vendor orchestrated by Zigbee2MQTT and while some devices are quirky sometimes (worse offender... IKEA) I'm not planning to move over to Matter any time soon at all.
> That said, it is noticeably slower than Zigbee, but we’re talking 50ms > 250ms. Which is more than acceptable.
I’m taking your “but” to mean “it is acceptable.
I would at least deem it noticeable to wait a quarter second for my light to turn on. May just be my own impatience, but I get an insta-headache whenever I have to wait to find out whether something worked or I have to try again. Every bit of jank is a downgrade in quality of life to me.
Now: May this be due to that specific setup or is the delay somehow inherent to the technology?
In many situations you're already "latent" due to interacting via voice: "Hey Siri, turn on the lights" is ~3-4 seconds to say casually, allow another 0.5-1.0 seconds due to voice processing, network, etc, and the missing 200ms is basically par for the course.
Where it starts to get annoying is "door open" and "motion detected" triggers. Motion detected can be played off as a sloppiness in the sensor cone, but going from "instant" to "laggy video game" when you have a physical trigger can be relatively annoying.
A lot of graphs where OpenThread & zigbee dance around each other, are within 2x of each other.
Two huge notables to me: message throughput for openthreads scales up, zigbee seems to hit a wall early. Not sure when that matters but very clear. The last table was a fright through: it takes half a minute for OpenThreads to recover after a node drops where-as Zigbee a quarter second! Wow.
Is there a situation in which your smarthome taking 30s to recover from a node failure is impacting your life? Seems like a distinction without a difference
Do you have any smarthome stuff? Standing around waiting 30 seconds for your light to come on would be moronic. A complete regression over the electric switch invented in 1884. I'd rip everything out right then and there. Your home should be reliable, smart or no.
Whilst I agree a smart home offers some benefits that your 1884 switch simply cannot match. And the cruicial point is that the 30 seconds are the worst case situation where something went wrong on the network. Its not the usual operational experience you'll get.
For me, having to wait 30 seconds once every blue moon is a tradeoff I'm willing to make.
Buttons etc. have direct binding. So even if your hub goes up in flames those will just keep working.
The automated stuff like "starting at sunrise gradually adjust the lights" might be delayed by 30 seconds. Annoying but hardly rage inducing.
Standing outside with an armfull of groceries, and a door is now our of range of your hub temporarily due to the failed node. 30s can seem like quite a while standing in the rain.
Not exactly a strong argument if you need like 5 qualifications to demonstrate the issue. If the right node fails, if it happens right as you walk up, if it's raining, if your porch is uncovered, if your door doesn't have an immediate fallback... I'm sure someone will be affected because of the law of large numbers, but it seems like an unlikely scenario to base a purchasing decision on.
A light switch not working for 30 seconds, or a door not opening for 30 seconds, even if only sporadically, is a significant loss of life quality, no qualifiers needed.
> A light switch not working for 30 seconds, or a door not opening for 30 seconds
I dunno if its just me, but having a manual override for such important/basic stuff is kind of a must for me. As much as I enjoy smart home stuff, having anything critical only available via that is just stupid.
Because IOT devices are cheap and networks are ad hoc, flaky connectivity is a likely occurrence. Thread must be reliable in these situations.
`establishing it as a more robust foundation for larger and more heterogeneous deployments.`
Based on.... 6 devices?
Quite honestly I would have been interested in measurements with more than only 6 devices. Maybe upto 50 or so,even better if they'd mixed different hardware nodes for a more realistic scenario
I have 47 thread/matter devices, alongside some 10-15 matter over wifi devices, so not quite 50, but "close enough".
It's not as fast as Zigbee, not in day to day usage, and not in terms of reconnecting, which often takes 30 seconds (ie. after a power outage) where my Zigbee devices just seem to reconnect instantly. On day to day usage there's a slight delay when turning something on, but it's maybe 100ms or less.
I initially had quite a few problems, but after "standardizing" on a single thread network (I had 3 before for some reason), as well as adding some wired thread border routers, things have become much more stable.
My zigbee to thread is still a "work in progress", replacing devices as they fail (ie. lightbulbs), and I'm also still adding new Zigbee devices like the Aqara T1 Valve controller, but I prefer thread over zigbee today.
One thing I will note, people tend to underestimate the range of thread (and zigbee for that matter). Each mains powered device will function as a repeater just like zigbee, and I initially thought I might need a couple of smart plugs as repeaters, but despite having a somewhat large house (by european standards) as well as double brick walls dividing it, my thread devices have no problems "seeing" a thread border router in the opposite end of the house.
Thank you for giving some insight on your experiences I practice with that amount of devices.
I have a zigbew network running with around 55 devices current. Ever since matter over thread I'm wondering whether I should start making the switch.
Tell me, what made you start migrating to Matter? General availability of hardware, latency, stability, something else?
Initially it was just curiosity when thread/matter was new. Back then it was a horrible experience, but I was using primarily HomeKit as my home automation platform (which only supports thread) and didn't have that much complexity. I was also using a bunch of vendor specific apps to control each individual vendors ecosystem.
Fast forward some years and I had my share of vendor specific ecosystems be shut down, leaving me with a bunch of technically fine paperweights, which was when I started focusing on local first access. I didn't want to be locked in by yet another vendor, and I was still using HomeKit as my main platform (still not very advanced home automation), so thread/matter came naturally.
What really turned things around was when I purchased a Homey Pro in 2022/2023. I could integrate pretty much everything locally, and expose it to HomeKit to the "control surface" my family used, which is when I started using Zigbee more (only had Hue before that).
These days I'm running Home Assistant. The Homey Pro became too small, frequently running out of memory, and switching to Home Assistant opened a whole new set of doors with ESPHome and others, like gaining control of my Vaillant Heat pump thought an ESP32 [1], or reading the wmBus telegrams from my water meter.
My "main criteria" at the moment is not so much if it's thread, zigbee, or wifi, but heavily focused on local control. I'm only buying devices that have local access. I'm fine with devices offering an optional cloud plan, like my Tado X TRVs, but I must be able to access them locally from HA.
I still default to thread/matter where possible as to not have too many things going at once, and I have zero Z-Wave devices.
Absolutely. My setup which is about 1 month old already has 14 devices and I'm just getting started. Only having 6 devices really feels comically small for a study investigating the performance of communication protocols where device count can really make a differnce, as observed by even then changes from 1 to 6 devices.
I’m a bit surprised they didn’t add zwave into this comparison.
They're probably only comparing open protocols. If one doesn't care if their protocol is proprietary then sure one might consider zwave. But it's hardly "surprising" that it's left out.
The funny part is that people think that Zigbee and Matter are open :)
All three of them have their shortcomings. Matter requires an unsecured IPv6 network to operate; I have my doubts about it having any longterm future.
> The funny part is that people think that Zigbee and Matter are open :)
How is Matter (APL2) over OpenThread (BSD-3-Clause) not "open"?
> Matter requires an unsecured IPv6 network to operate
How is this network "unsecured" exactly? Thread IS encrypted and authenticated.
What? The entire specification is in this git repo maintained by the home assistant team. Also, they’re on the zwave board now. It’s wholly open, but wasn’t maybe 4-5 years ago.
Source available is not the same as open. Source available does not mean non-proprietary.
Just like HDMI, if you want to build a device that uses Zwave, you must pay for the privilege.
I'd like to use z-wave but as an Australian we have different allowed frequency bands (specifically the USA-market gear uses frequencies that are reserved for telecom use over here, and anything that can interfere with the ability to call emergency services is Very Illegal), and thus the few devices that get built supporting the AU rf bands are overly expensive due to the (lack of) economy of scale.
As a result my home automation stack is a mix of zigbee, Matter (over both Wifi and Thread), and just plain WiFi. All of which uses the 2.4Ghz (and 5Ghz but mostly 2.4 for the IoT stuff) ISM band(s) and is thus cheap and plentiful.
Not sure how many other regions suffer the same issues, but I suspect that z-wave will be outcompeted by the more modern alternatives in the coming years.
/rant
yes, and also that if someone is buying stuff new, it should be exclusively zwave800 stuff. example hardware on the "hub" end would be:
https://www.home-assistant.io/connect/zwa-2/
https://www.getzooz.com/zooz-zst39-z-wave-long-range-usb-sti...
the zooz is this internally: https://www.silabs.com/wireless/z-wave/800-series-modem-soc/...
another protocol/wireless stack? Why can't we settle down to something that is at least "good enough"?
Because zwave is proprietary. There's money to be made here! Can you imagine the lost profit potential if people standardize on an open protocol like ZigBee?
It is so proprietary in fact all of the spec is on github:
Even with a spec on github, a protocol can still be proprietary, e.g. if companies have to pay to use it legally in products
Try to build a device and not pay the tax stamp.
You can see it in products like inovelli where zwave is always more than zigbee.
https://inovelli.com/collections/z-wave-light-switches-red-s...
Do you really need long antenna? I noticed zwave can act as a mesh.
Most likely it's not a super robust transmission mode like LoRa (that works down to -145dBm). Having a well designed antenna with a good pattern help a lot then.
Doesn't zwave use a different frequency, and zigbee and thread are both 2.4ghz? Perhaps that makes it a more fair comparison?
Yes, zwave is in the much less congested 900mhz range. It is a fair bit more reliable as it doesn’t compete with:
- WiFi
- Bluetooth
- microwave ovens
Additionally, it’s a bit more strict with certification so you can easily use multiple vendors for zwave whereas with zigbee it’s often a terrible experience since vendors implement things differently in ways that are not as interoperable. In fact, for a device to be certified by the zwave alliance, it must pass strict conformance testing to ensure backwards compatibility and interoperability.
This is both good and bad, and no surprise its an American standard. Lower frequencies have higher range, which is good if you live in a house (because range), and bad if you live in an apartment (because interference and crowded channels). So it will compete with itself, and everything on the ISM band (which you have no control over).
All it costs is significantly more money and less options.
If you limit your pool of zigbee devices to just more reputable companies, you'll end up with around the same amount of options as zwave and they'll be reliable but significantly cheaper.
You also have the option to find some super cheap stuff that works just fine. That usually requires some research but not much. You can also build ESPHome devices for zigbee but that's pretty niche, to say the least.
Having previously been all zwave then moved and tried to go matter but ended up with zigbee, I wish I never listened to people saying what you are. As far as I'm concerned, zwave doesn't know it's dead yet and Matter isn't going to fulfill it's promises because the corporate interests that wouldn't give you local control with zigbee aren't suddenly going to change their minds because it says matter now.
My entire “smart” home setup is built with Matter over Thread. It’s fully local. No internet access required. With Home Assistant onboarding can be done without a smartphone now. Devices without official certification work but probably only in Home Assistant.
My Matter over Thread stuff keeps occasionally falling over in Home Assistant and becoming unavailable, probably a router issue as I’m using a homepod mini but still endlessly frustrating.
I’m using ZBT-2 with HAOS on RPi 4 4GB. It’s rock solid in a crowded apartment complex. Devices are 90% IKEA and 10% Aqara. Definitely try that antenna and don’t connect additional border routers. Let HA own everything.
My whole setup is, suboptimal, to put it delicately. It’s just running on a NUC that’s crammed inside my AV cabinet, antennas would be crammed in there as well. I mostly buy zigbee stuff but it’s hard to ignore how cheap the Ikea Matter over Thread kit is.
Dunno, personally I have found HA + MoT to be strictly worse than Z2M. It's the classic case of the Douglas Adams quote:
“The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair."
HA MoT has a simplified UI, that hides hellish complexity underneath, that's held together by loose pieces of gum, which means its difficult to troubleshoot. HA has a rather laissez faire policy to stability, for example, they rolled out an update just recently that made it impossible to add new devices.
I just readded yday evening my Ikea temp sensors as they mysteriously stopped working for no reason. Why? Who knows, but you can't ship stuff like this to a non-technical audience, especially if they rely on it. I had it connected as the temp sensor of my AC, and was surprised to find the AC was not working, thats how I found out it suicided itself.
I never had any problems with Thread that couldn’t be fixed by pulling batteries from a crashed Ikea device and I have very little interest in how the protocol works.
Automatic updates in Home Assistant are disabled by default for a reason. It’s not an ultra polished, safe corporate environment for non technical wives and children.
Which sensor? Battery powered temp/humidity sensor or USB-C powered air quality monitor? I’m asking because the battery powered one is known to be bad. They disconnect and refuse to connect again until their batteries are pulled for ~10 seconds. It’s a device problem, not Thread problem.
But Matter _is_ locally-controlled. Matter-over-Thread is better as there's no side channels to abuse; Matter-over-Wifi also guarantees local control though the presence of Wifi means a high probability of some additional onboard software bloat the manufacturer can use to enshittify the product if you're not careful.
On Europe the 860MHz band is subject to more restrictive transmission duty cycles, LBT or FHSS.
https://en.wikipedia.org/wiki/Short-range_device#SRD860
In the US you also have the FCC reshaping the 900 MHz for industrial broadband and NextNav. So there will be more competition.
https://www.nemko.com/blog/fcc-reshapes-the-900-mhz-band-for...
https://www.landisgyr.com/br/pt/home/knowledge/blog/Navigati...
Zwave frequencies, as well as regulations, vary between countries/regions. This means that equipment isn't interchangeable, something I'm currently running into - I can't get a zwave electronic door lock for Japan while I could easily have bought one in my home country. But I wouldn't be able to bring it if I wanted to.
Zigbee is basically the same everywhere, however the 2.4GHz band is of course somewhat congested.
some zigbee controllers even support being converted to thread via a firmware update
These protocols add so much complications it makes wifi/tcp/ip a blaze
They fix some problems but add more, you endup just like that xckd about protocols
Having switched to zigbee devices recently I regret trying to start with WiFi. While I'm not sure how it looks under the hood, adding devices with home assistant was miles easier.
I've helped lots of people with home automation Network build outs l; the biggest factor is the RF environment.
If you're fortunate enough to not be battery constrained at most endpoints and can put a high quality Wi-Fi access point in an optimal location than Wi-Fi is incredibly reliable.
The tooling for inspecting all layers of a Wi-Fi network is infinitely more accessible and mature and that certainly helps; I can't go to My local big box store and get a USB thread dongle for 20 bucks that will just work on a Lennox computer and integrate nicely with Wireshark out of the box...
No, you have to order it from Mouser for $16 instead.
Fixed link: https://www.mouser.com/en/ProductDetail/Nordic-Semiconductor...
And there’s a little configuration: https://wiki.makerdiary.com/nrf52840-mdk-usb-dongle/guides/n...
I have a Unifi home network with two Wifi 7 access points and gigabit ethernet. For home automation, I went with ZWave and do not regret it at all. It's been rock solid and I've never felt the need to get a dongle to debug the traffic. I don't want to be working through all of the WiFi baggage with my home automation devices. I recently introduced some Zigbee bulbs (hue and cheapos from Amazon) and they've been mostly "fine", albeit with lower range.
Thread is fairly similar to Zigbee, just standardized and doesn't require brand specific hubs. You can actually control thread devices direct from an iphone with no hub at all.
Wifi for anything but cameras is a mistake, it only exists for maximum compatibility since everyone has wifi already.
Wifi for cameras is arguably also a mistake
Cameras that require internet to connect to are a mistake, but cameras that use Wifi for local link are fine (wifi is perfect for this). And, having some way to connect remotely is a requirement, for most everyone.
I took their point to be about wifi being jammable
Yeah first thing burglars are gonna do is jam your wifi camera so you don't have any video of them. Run cat-5 and get a PoE camera.
And be sure your outdoor cameras are on an isolated VLAN so people can't just plug in to your home network.
I read this as sarcastic but it's a thing. Simplisafe, for example, is trivially jammed.
Yeah, not sure why people are downvoting you. I suspect because most people run wifi cameras and don't see the reason to run a ethernet cable.
If you are halfway serious about your home security, you should use ethernet. It's trivial for thieves to jam wifi now.
Security system that can be jammed is a terrible idea. Cameras should be wired. Ideally on separate network and with battery backup.
> Wifi for anything but cameras
Buttons (key presses, remotes - think rapid input), Knobs, sliders, Pixel LED' strips, some MM wave sensors, ir/radio transmitters, really need wifi.
There is products where you will want a bluetooth proxy, plenty of kitchen/grill thermometers and scales for kitchen and bathroom are great examples.
Where range is an issue LoRa is a thing that has plenty of devices and is easy to DIY.
My matter over thread devices (inovelli) for light switches work faster and better then my old wifi devices (cync)
A bit of clarification: Thread is a layer 2 protocol, for the user, it looks like standard IPv6. Matter is application level, and just expect standard TCP/IP underneath. There's nothing binding Matter to Thread, and in fact a lot of Matter devices use Wifi
I studied zigbee protocol for an exam and while everything is there for a reason, it truly feels diabolical
Thats exactly what Matter over Thread solves. It just uses standard ipv6, udp, tcp.
WiFi is not designed for lots of low energy devices that can be repeaters.
I'm running both now, and while my Matter network seems to be solid, I can't get over the fact that it's so much harder to set up devices than on Zigbee. Why does Google need to get involved for me to set up a local IKEA device with my local border router?
As someone with hundreds of Zigbee devices w/ zigbee2mqtt in a network (actually 2; one in my standalone garage, and one in the house), I'm hoping Thread/Matter will not take over because _for me_ it does not have a single improvement over Zigbee 3.0 and I have had 0 inclination to adopt any Matter devices so far. Probably why Zigbee 4.0 is a thing.
I have tons of devices and I wish everything was Wi-Fi.
Wi-Fi has its problems but at least it is easily measurable, debugable, ... Zigbee is a voodoo protocol: you initiate the pairing and hope for the best. And then you hope for the best to have it stay on.