How Apple designs a virtual knob (2012)
jherrm.github.io166 points by gregsadetsky 5 days ago
166 points by gregsadetsky 5 days ago
> After using the knobs in Garageband for a while, I noticed that they didn’t always react the way I thought they would. Most of the time the little indicator dot on the knob would follow my finger as I spun the knob around in a circle. Other times the knob wouldn’t follow my finger at all and seemed to go in random directions. I eventually figured out that I had stumbled on three different ways to turn a virtual knob.
> ...
> Apple’s attention to detail is what has propelled it to be the most valuable company on earth. Whether it’s the click of a physical button or the math behind inertial scrolling, Apple employees work really hard to make products that are deceptively simple and just feel right. The virtual knobs found in Garageband are no exception and I hope others enjoyed learning about them as much as I have.
I think these two statements are contradictory. Personally, I've noticed a pattern when people post about Apple UX that seems to go "yes this thing may be unintuitive but actually it's a sign of really good design!" that I can't quite seem to wrap my head around
I think it's more that someone may assume how something works, and it isn't exactly that, so they say it's unintuitive. But there could be multiple assumptions on how it should work on first use. Covering all of those possibilities, and integrating them into a cohesive experience that works the first time, and makes even more sense as you continue to use it and learn the other ways to interact, shows a strong attention to detail and design.
This is opposed to something that may be very intuitive for 30% of people, but the other 70% are lost, and the implementation doesn't scale.
I think in UX there is general lack of desire to properly explain how stuff works instead of relying on just "guessing user expectations right"
like if said knob just displayed a vertical bar with marks signalling up and down also works it would be very clear to person that tried to just spin it
The problem of the knob is that it offers a large and precise control, but that large control remains invisible. There's no obvious clue showing that you can still interact with the knob by dragging the pointer / finger far away from it.
Adding a simple visual clue would help discoverablility a lot. Draw a faint halo on touch, when the mode changes. Draw a more visible trail when the touch point is dragged. Provide immediate and localized feedback, like good UX guidelines suggest.
Exactly. Based on this thing's depiction as a knob, every user is going to assume that you have to describe a ring around it by dragging the cursor in a circle... a fussy and awkward pain in the ass.
"Apple’s attention to detail"
Such cringe-inducing, delusional fawning. You can find counterpoints to this oft-regurgitated claim all over Apple's products, with a cornucopia of them in the rightfully-scorned Tahoe release alone.
But back to the topic at hand: "knobs" in GUIs. They suck, for the very reasons demonstrated here. Audio software in particular is replete with this skeuomorphic failure, and why? Because people who work in music or audio can't understand more-effective GUI affordances? Because they'd be lost without something that looks like the physical knobs on a mixer or stompbox? What an insulting assumption, not to mention nonsensical in modern times.
"Apple employees work really hard to make products that are deceptively simple and just feel right."
This grand declaration is based on what, exactly? In a decade as a software engineer at Apple, I saw a wide range of dedication and aptitude in UI design and implementation. This varies within teams and between teams, with no set standards for research or testing of UI effectiveness. I saw the same amateur-hour mistakes made repeatedly, despite their being pointed out incontrovertibly... and some have come back to bite (and cripple) new generations of Apple products.
Design isn't getting better, folks. It's one thing to give bad design a free pass; but to LAUD it hurts all users.
The idea of local maximum means to get from one local peak to another, even if it is a higher one, you need to first go through a valley.
Somebody comes up with an acceptable solution to a design problem, people get used to it, then when somebody comes up with an otherwise theoretically better solution it is in practical terms worse at first because the user has an intuition for another one already. Having to relearn is worse than if there was no preexisting intuition yet.
This is why it is possible to say it is both not very intuitive but good.
ease of initial understanding is not the only definition of good UX. Some UX is optimized for one time use with no familiarity (wizards work well, like checkout flows). The opposite is designed for use every day. In real life professions it takes a lot of work to learn to use tools effectively. That’s not because they are poorly designed, but because they target optimal performance after an initial learning period.
Software has many successful examples of the latter but it’s not a paradigm that designers are familiar with or that startups tend to go after.
author here - fair point but I still think apple made the right call even if it leads to a bit of confusion at first.
If a digital knob needs to be turned several times (e.g. 1080º, common in DAWs), the "default" way to interact with a knob on a touchscreen - circling again and again - is slow and uncomfortable. Adding "slider" gestures on top of the default behavior is a nice way to perform many turns quickly and easily.
Reminds me of the Steam Deck's touchpad for scrolling. Some people are very confused by the default behavior which seems to scroll up and down randomly when you swipe. But for people who used an iPod it makes perfect sense, and you can scroll continuously by moving your thumb in a circle, rather than scrolling in increments by swipe swipe swipe.
If I had to guess, I think he may be trying to describe the behavior when you try to interact with the virtual knobs in a way that is based on assumption of how it would work due to other experiences in digital UIs, rather than the “level of detail” that Apple puts into UI and UX that is at first contrary to experience, but then actually amazes people when they realize it actually works with their human expectations rather than the human having to have to adjust themselves and their expectations to the poor UI and UX interactions of other experiences.
To be more concrete, if I had to guess, the author tried using the knob like a slider, trying to drag the visual slider down to decrease and up to increase the knob rotation, but that conflicted and caused movements in “random directions”.
The whole idea of knob is stupid both on touch screens as well as desktop. There are other good alternatives which are far more intuitive than knobs.
Knobs are good when you can physically rotate them like for example in a car. But there we are removing knobs and adding touchscreens.
In just about every simulator featuring knobs, I've noticed that most knob interfaces will accept scroll wheel inputs. Use the literal knob in your mouse to control the knob on the screen.
Of course Apple's mice don't have a physical knob, so that approach doesn't work, but knobs and mice can work outside of the Apple sphere.
On touch screens, you can probably make them work by tapping the knob and popping up a slider to control the value. Lets you use knobs to maintain an overview while exposing usable controls for modifications.
I’m amazed you actually think that Apple’s mice don’t have any way to scroll.
They have touch area where the scroll wheel would normally be, that works extremely well.
The point of knobs is that you can fit a ton of sliders in a limited space, and that you can wildly adjust them with very little movement. Both are requirments for a lot of music software. What would the alternative be?
> What would the alternative be?
Large slider which doesn’t change place, buttons to select what you are adjusting. Display the current value on the button if you need it to stay visible.
The magic of software ux is that you can actually replace things on a screen in a way you can’t on a physical device.
Then you can only adjust one thing at a time—so you’ve just created the worst of both worlds with a multi-touch display and live music software.
There is absolutely no way you successfully adjust two knobs at the same time on a multitouch display, let alone while doing live music. They are barely usable one by one.
There is a reason people serious about doing music keep using physical knobs to change values in their software. I’m entirely convinced the sole reason DAWs use virtual knobs despite them being such a poor UX element is because people will map them to MIDI knobs anyway and that keeps the software and physical world looking the same.
> There is absolutely no way you successfully adjust two knobs at the same time on a multitouch display, let alone while doing live music
I do it all the time on an iPad. It handles up to 5 simultaneous controls very well.
There's a demo of the original iPad multitouch display, which can distinguish eleven simultaneous inputs.
Prompting the obvious question: Why eleven?
Kind of a self fulfilling prophecy then. I had to figure out how to map the knobs to MIDI hardware because it was so hard to adjust them on screen!
At that point I guess the physical resemblance is a virtue.
How about a knob that instantly overlays a slider when you touch it?
Then it takes only knob space at rest, but offers the slider affordance and high-res control when you need it.
Maybe that would work if it popped up VERY FAST. I have never gotten along with touch screen knobs.
Some of the music production UIs seem to feature huge grids of knobs, a pop-up slider would inherently obscure some of the adjacent knobs in some way, requiring you to move the cursor away or click somewhere else to dismiss it. It would create friction if you needed to do quick adjustments across a row or column of knobs.
I think the best compromise is something that's already very similar to knobs - a "draggable text field". Different software styles this element differently, but the essence is that it's a number, where you can either click and type a new value in, or hold the mouse on the field and drag it left or right to drag through the different values. You can find this in some video and 3D editors. Sometimes these elements are styled to have arrows on the sides of the numeric value to suggest the dragging behavior.
That's effectively the same as the Apple knob modes where you can drag vertically or horizontally, except the visual slider would be locked to one orientation.
There are some music software that do this, and it looks clunky shifting between a graphic of a slider when you're moving it and a graphic of a knob when you're not.
The difference with the Apple knob is that you would be given a strong cue for what to do after you tap on the knob. The current knob design has no cue until you press and slide, when you see a value change. But you have to do it right before you get any cue. Real knobs are turned; not tapped, pressed or slid. The slider shows you an affordance that works. The knob shows you an affordance that does not.
I think multi-zone drumpads on the recent Akai MPC Live 3 provide a good middle ground, quite similar to mapping various zones on a trackpad. The Macbook touchstrip was a cool (but maybe too cool) addition as well, similarly introduced by various DAW controllers (Push, Machine, MPC Live, others).
Look at the how timers are adjusted in the Clock app on iPhone. Three rotary tumblers that can all be set independently.
Designing 3D real-world interactions for 2D screens is fun. Literally fun. Rarely useful.
Yeah, the paradigms are just too different.
I prefer sliders for knobs… just much more natural with a mouse or touchscreen.
It's hard to replicate the "coolness" factor though of a true studio control board. It begs to be touched and knobs beg to be turned...
Completely agree. They are very prevalent in DAWs and audio plugins, as they try to look like physical hardware. I absolutely hate interacting with them, either with touch or mouse.
I guess the one advantage they have is they don't take up as much room as a slider, maybe?
I actually think knob inputs i.e. just the knob without vertical or horizontal modes, are quite useful. The ability to naturally gain precision the further out you drag is very handy and intuitive.
Not good for computers with mouse inputs, but for touchscreens I like the idea.
>The ability to naturally gain precision the further out you drag is very handy and intuitive.
Pie menus, where the selection is based on the gesture direction, allow you to move further out (longer gesture) to get more "leverage" or precise control over the angle (either continuous angle, or the selected slice).
The angle selects a slice, but you can think of a knob as a pie menu with one slice (the whole pie) that also has a direction and a optional distance parameter.
But you can even use the distance to exaggerate the angular precision even more!
Here's a demo of a "Precision Pie Menu" I wrote in 1988 for NeWS in PostScript, which exaggerated that angular precision effect even more, once you pass a certain distance, allowing you to have extremely precise control over the angle.
https://www.youtube.com/watch?v=c0scs59va4c
>Demo of the precision pie menu. Research performed by Don Hopkins under the direction of Mark Weiser and Ben Shneiderman. Developed and demonstrated by Don Hopkins.
>Transcript:
This is a demonstration of the Precision Pie Menu under the NeWS window system.
It's an experiment in exaggerating the extra precision that you get with distance.
As you move out further from the menu center of a pie menu, normally the further you go from the center the more control you have over the angle.
But if you want to input an exact number like an angle, you might want to get it down to the a certain number, but you run out of screen space before you get enough leverage to change the number to what you want.
Now what happens here is that when you poke out, it makes a flexible lever, that the further out you go, the more flexible it becomes, and you have much finer control over the number.
So as I move around back in and out, I'll poke it into a different place and just come out further to get a lot of leverage, and dial exactly the number I want.
So here's what happens when you go around to the other side: "pop pop"!
And as you get nearer it gets less and less flexible.
Generally you'd kind of eyeball it, and then get it exact like 93, well there's 93 or 273, there's 273.
It's really not. You're looking at it the wrong way, and haven't come across any of the use cases it solves.
If you have limited space and you need to both interact with and see fractional ranges, knobs are the way to go. It's way more glanceable, and the entire range is displayed in the knob itself.
Think of it this way: Both a circular knob and a slider have 2 elements: the interactive area and the range display. However, the slide has the same knob size that is set on a large track displaying the selected range by moving the knob, whereas the circular knob has the track displayed radially inside it.
For the track example — the knob is the only interactive element for all practical purposes when it comes to precise tweaking of values. Single clicks on a track usually don't support further dragging after the initial click on any OS or UI implementation.
This comes with many positive sides:
- The interactive area (handle) is always in the same place.
- The interactive area is in practice always bigger than a knob on a linear slider.
- Adjusting the knob doesn't reposition your cursor, no matter what you do with the mouse.
- The circular track allows for much easier visual identification of fractions compared to a linear track due to its radial nature.
- The indicator can be a single pixel, whereas on the linear track, the knob is a fairly imprecise blob due to its nature of needing to serve a dual purpose. This means it's a lot more precise.
- There is a lot more granularity in the same surface area.
- Interaction precision isn't limited to the size of the track where it needs to scale linearly
- You don't need to dynamic element rendering or resizing which may cover other things you're looking at.
- The area is much smaller. On a 16x16px circular knob, I can get up to hundreds of steps which are clearly visually distinct.
All of that being said, the article is quite bad as it contradicts itself, and uses knobs in ways they are not good at, which is circular interaction and being able to do multiple circles. It beats the point of having a knob, might as well have an interaction handler on the number indicator itself.
If you've ever used pro audio software you come to love rotary over linear sliders. They're simply more flexible and dense when you have many parameters to tweak.
Linear sliders are also finite, while rotary encoders can spin forever.
Yeah but that doesn't have much value as you lose the value indicator.
If you don't need a value indicator, you don't need a circular knob as an affordance. You can have a whatever as it just reacts to your input.
If you do have a value indicator which is "infinite", such as a numerical value display, it's better to make it interactive and place the interaction on top of it, instead of splitting the UI between a value indicator and the input.
A lot of software does this.
In the physical world, sure, but there's no reason why a virtual linear slide could not be made endless
You'd have a discontinuity. A knob naturally rotates back to its start. A slider never does and would have to pacman-wrap itself back to the start.
There's no motion you could make that would infinitely increase without a break. A knob you can just move your cursor in a clockwise circle infinitely.
There's a lot of people complaining "Why use knobs?" and a lot more people giving the reason "Because DAWs use them" without explaining why DAWs use them.
First, it's a visual representation of the value, and it's easier to map "Slightly more to the right" than "an extra 0.7"
More importantly, all DAWs can map physical control surfaces to the on-screen knobs, and control surfaces all use knobs and sliders.
For example https://mackie.com/en/products/mixers/onyx-series/Onyx16.htm...
I'm amused by the contrast between Apple's attention to detail on the implementation and their failure to recognize that a virtual knob with a touchscreen or mouse is a fundamentally bad idea.
The author also makes this error, praising Apple's design prowess and denigrating its competition while failing to recognize they "didn’t always react the way I thought they would" because they're ill-suited to the medium.
Literally every DAW has knobs everywhere: it would be impossible to use sliders everywhere in a DAW's UI, there simply isn't enough room.
"Make [a slider] bigger while the mouse button is held down, and warp the mouse so that when you let go you pick up where you left off" has been a solved problem for decades.
And with traditional toolkits (i.e. not HTML) it will even be fast.
It's not all about the interaction, but also the visual representation which can be much finer and granular in small spaces with a knob.
I can make a 16x16px knob where you can see almost the entire 320° of the range.
It's also easier to see fractions, such as 1/2, 1/3 or 1/4.
Sliders, especially in 16px possess none of those.
I was able to debug and fix someone's MainStage patch last night over SMS when they sent me pictures of their screen, where all the knobs were visible.
Being able to see the full state of the thing is important. Hiding it behind interactions is just as bad as hiding it behind menus.
Sidenote, you have to do this on native because pointer lock/warp is not universally supported in web browsers.
author here - I made this comment elsewhere, but I still think apple made the right call even if it leads to a bit of confusion at first.
As others have pointed out, sliders have limits & knobs don't, so I do think they have their place on touchscreens.
If a digital knob needs to be turned several times (e.g. 1080º, common in DAWs), the "default" way to interact with a knob on a touchscreen - circling again and again - is slow and uncomfortable. Adding "slider" gestures on top of the default behavior is a nice way to perform many turns quickly and easily.
I'm curious - what UI mechanism would you use instead?
This is not an Apple original design, this is standard fare in DAWs and VST plugins and has been since at least the early 00s. In the beginning of the article he talks about context menus as something that is not one GUI's but just standard in the industry - these knob interactions are like that for the audio industry.
author here - I was specifically talking about digital knobs on touch screens having both spin gestures AND horizontal/vertical slide gestures.
If you can point me to a DAW besides Garageband on iPad which was on a touchscreen with those three gestures I would love to try it out!
Those gestures have been pretty standard with mouse movements in music software for a while. Apple brought it to touchscreen, but they didn't invent the multi gesture knob interface.
I had a Novation Zero SL mkii in 2008 which had a "universal knob". It acted as an HID mouse input and when you spun the knob it would click your mouse and drag it vertically. It worked with most music software at the time because vertically dragging a knob had become fairly standard.
QuickTime use to have a wheel as a volume control.
It’s was a pain to use and they later dropped it for a slider.
You beat me to posting this. When this version of the QuickTime player came out, I couldn't understand how Apple of all companies could ship this obviously awkward control.
Edit: Scrolling further down on the article, I get reminded of the weird pop-out drawer at the bottom of the player. I had totally forgotten about it, and it was also a very awkward and un-ergonomic piece of UI.
Huh, the knob turns back when you attempt to turn it circularly (the most intuitive gesture).
How difficult can it be to make a knob that works when turned both linearly and circularly?
The knob is clever and very well emulates a physical knob. That's not a good design for an on-screen one though.
- Going "around the rim" in an arc is difficult both for touch and mouse. Results in jerky motion.
- Defining the direction based on which outer rim I am closer to makes it unpredictable, especially for small knobs.
Best in my opinion is the "click / hold the knob and pull vertically or horizontally with up/right turning to the right. Makes it fast and predictable even with little space like a DAW audio mixer.
Up and down virtual knobs are entirely unintuitive to me.
I understand the rationalization, but a knob is not a slider and what's the point of non-skeuomorphic skeuomorphism?
This approach solves a common problem in apps that need to surface a lot of controls.
Problem 1: Sliders take up a lot of space.
Problem 2: Fine control of a mouse or touch-driven interface is provided by sliding, not by rotational gestures.
The idea here is to use a virtual knob to save space, while providing the fine control possible with a sliding interface. The sliding direction is generally chosen to be intuitive to the function of the knob. (Locking to horizontal or vertical also assists with fine control.)
Using a knob when using a knob doesn't solve the problem is poor design...then again, skeuomorphism is usually bad design.
Here a counter that increases and decreases with mouse movement would take less space and be more intuitive.
And a much much better design because it would provide a numerical readout of the value directly at the point of interaction.
But in fairness, most design is bad because designers tend toward satisfying themselves rather than users...ok, I will stop ranting now.
Just flat counter fields are terrible in audio software interfaces. Sliders and knobs give you visual feedback of where you are along a line and an easy way to quickly increase/decrease speed when adjusting.
Most software I use does still show some numeric value somewhere, either around the element when changing it, or in some other panel. This way you get some more information than in the hardware equivalent if you need more granular control. Its particularly nice if they allow you to click/double-click for editing values.
From my perspective as a user, knobs convey exactly what I need. Mostly I don't care about the exact number, just about what position something is in. Knobs behaving like sliders is fine. I'm not physically moving a knob, I might be moving it with a mouse or touchpad. You can't stray with a physical control the way you can with a digital one. And they allow interface designers to put a lot more information on screen where space is at a premium.
Honestly, just go download a trial version of something like Reaper or Reason and go make some music. You'll get a better feel.
It's quite common in DAWs, it allows you to adjust knobs quite easily with a mouse.
It's a fixed size slider which uses the rotation of the indicator to tell you its position, instead of the position of a thumb in horizontal or vertical position.
If you replaced it with text or a bar that filled the area it would be the same.
It's better than a linear sliders because it takes up less space. It's better than a bar slider because you have more range to display (the length of the arc of the indicator is longer than the horizontal and vertical dimensions). This in turn makes it better for putting into tighter spaces.
Your rationals don’t mention users and if you are short on space, that’s poor design — just as a 12” (300mm) bedroom door would be poor design in a dwelling.
By way of survivorship, every popular DAW and plugin responsible for music made over the last 30 years has made extensive use of knobs. If it's poor design that's news to the industry.
Like if knobs suck, then every audio UI has always sucked.
The main interface in Garage Band uses sliders…same with Pro Tools.
Because competent designers.
I don't find these knobs any less irritating then the knobs in other music software. Sliders are the perfect way to change numbers with a mouse or a touchscreen. I don't understand why music software sticks to a level of skeuomorphism that has been abandoned in every other field.
It is interesting how opinionated people are. Personally, I think we just don't know enough to do a reasonable assessment. If we had some example of where a knob might be good and then try to understand whether it was good and if not, why not?
One of the issues (in my opinion) is how much control per pixel you get from a control. Certainly a knob has more control factors than a button. If nothing else you can click it and turn it. In the same size, a knob has much more control effect.
The other issue is how easily and comfortable a person can be using the control. This is complicated. If you see something on a screen that looks like something from the real world, then you have an idea of how to use it. And many of us became very comfortable with the old goofy Windows controls. But initial comfort is not necessary. Learned comfort is. If initially we don't know what it can do and how to use it, can we learn and once we do will we be comfortable?
It is not straight forward. The challenge I face is using a very small form factor - a mobile phone screen - to inter operate with complex systems and vast amounts of data.
The fact that it embodies more control factors in a small package makes it interesting.
Having played a lot of MSFS 2020/2024 recently, I feel like I can appreciate this way more now. Since they have to make these knobs realistically and in 3D, when using them with a keyboard and mouse (or even worse a controller) it’s incredibly difficult to see and turn them. It gets even worse since you can push and pull many of these knobs (the difference being potentially catastrophic as well).
A circular man proto I put together a little while back https://codepen.io/theprojectsomething/pen/JjmgXrg?editors=0...
I love how it’s handled in design software. Drag on a numeric value and it adjusts. I think it works the same in some DAWs for cases where you can display a numeric value / note / etc, but I guess in many cases knobs don’t have that and it’s some abstract percentage thing instead?
The one called garage band synth knob with 17 images is available as MF-A01 in real life. But beware there are 2 versions with same model number get the one with the set screw and brass bushing.
It gets very glitchy if the pointer/finger is near the centre of the knob. Really that area should be disabled or "worked around" somehow. The hard part is you can't stop the user's finger/pointer from crossing through the middle, like a physical knob does by its physical construction. So there's a mismatch there.
Pretty interesting- on the first knob (with vertical and horizontal disabled) works great with how I thought the horizontal and vertical gestures were supposed to work - the difference being I did them on the edges instead of the center.
I found this knob to be the best experience.
Curious if others feel strongly for the centered experience.
Ironically, this post perfectly demonstrates up why these gestures should not be used together. I could not reliably make it trigger one vs. the other and which mode it selects is not something the code can detect without continued input which will lead to discontinuities in value.
It would be nice if these had a 4th way of manipulating: focus + up/down arrows. They’re pretty inaccessible as is.
oh wow this is my article! wild to see it pick up traction after 13 years.
I guess I should probably publish more of my drafts...
Thanks for reading/commenting!
I don't know who they think they are fooling but these are all garbage. Jerky and frustrating to use, same as always.
sad this attention to details is gone
sad this attention to details is gone
This year so much has gone to shit in Apple's OSes (Tahoe and the other version 26 ones) it's unbelievable.
Did some CxO let their brother-in-law's cousin's nephew have a go at managing all the teams? I haven't felt this kind of frustration with an operating system since jumping from the Microsoft ship during Windows 8.
Some basic UI is literally unreadable on the dumbass "glass" implementation. There are blatant rendering bugs and placeholders still in the shipped version, just look at the effing Contacts app. DRM slowdowns have crippled the Music and TV apps so much I literally cancelled my subscription and went back to piracy. I'd post example screenshots but I already wasted enough time just writing this ragecomment.
fanboys are out of this world.
i can guarantee the only reason there's 3 input types for the knobs, is because three different teams did their own thing and nobody cares.
I didn't read the writeup. The result was pretty gnarly. The active area on a phone left me scrolling up and down and I had to go very slow once I got purchase on the knob or it would rotate back after a quarter turn.
Please no.
[dead]