"Self experimentation might be useful, but only if you understand the metrics you measure and whether its stochastic noise or meaningful results."
I feel you miss the whole point of self experimentation. If your aim is for improved mental function, more energy, less health problems, etc., you don't need to know how to interpret any obscure tests. You'll know.
A low-risk food product (such as this) can simply be trialed in a n=1 trial. If the results are positive, great. If not, go back to what you were doing before.
All the studies in the world only suggest how something might work for you. It is only the n=1 test that matters.
> If your aim is for improved mental function, more energy, less health problems, etc., you don't need to know how to interpret any obscure tests. You'll know.
Perception is different from reality, as exemplified by an Apple anecdote:
>This contradiction between user-experience and reality apparently forms the basis for many user/developers’ belief that the keyboard is faster.
Interesting article. I'm sure the $50M in studies were of non-power users, though, and so irrelevant for developers with sufficiently powerful keyboard shortcuts.
I can guarantee it doesn't take me 2 seconds for my top 20 keyboard shortcuts, any more than it takes 2 seconds for me to hit "return" at the end of a line, or to move around with arrow keys. That said, I frequently use the mouse as well, and always prefer to work in a GUI instead of a text-mode editor.
What I found most revealing, though, was the admission that part of the problem was the awkward location of the command key. Whenever I'm on a Mac I end up remapping it so that I can hit Control (in the corner of the keyboard) -- Control-ZXCV are all easier to hit than Command-ZXCV.
Which just goes to prove that most studies are wrong [1], or at least can't always be generalized.
>In fact, I've come to realise just how appallingly bad most of us are at using our tools.
I don't disagree. I know developers who use Notepad to develop, which boggles the mind.
In fact, I don't know why I bothered posting. I'm enough of an outlier in keyboard performance that, even if I'm right, it proves nothing. I've had many developers' jaws hit the floor as they watch over my shoulder. I've even shocked vi and emacs users into no longer trying to convince me their ways were better.
Maybe I should teach a class on how to use a GUI efficiently.
>I'm sure the $50M in studies were of non-power users, though, and so irrelevant for developers with sufficiently powerful keyboard shortcuts. I can guarantee it doesn't take me 2 seconds for my top 20 keyboard shortcuts, any more than it takes 2 seconds for me to hit "return" at the end of a line, or to move around with arrow keys.
Without measuring it A/B I cannot guarantee anything. Those people were just as sure. And I don't think they were keyboard un-savvy -- at that point in computing, they were far more proficient with a keyboard (AppleII etc) than a mouse.
>Without measuring it A/B I cannot guarantee anything.
You don't need to guarantee anything; I know my navigation and coding are way faster my way than if I were to ignore my shortcuts.
>And I don't think they were keyboard un-savvy
As a sister comment to your mentions, MOST users, even developers, are still extremely limited in the number of keyboard shortcuts they use. I'm an outlier.
If I want to save, my fingers hit Ctrl-S before I could even FIND where the mouse pointer is on the screen. Same with Ctrl-A for Select All. Or Shift-Ctrl-Left for select the previous word. No A-B testing required; it's seconds using the mouse for any of those, and 300-700ms for the keyboard.
And those aren't even my most powerful shortcuts; they're just the ones ANYONE can use on Windows. If you get into "complete this word from [above/below] in this file", "jump to definition of this function" or "show references for this function", or "jump to previous bookmark", or "set bookmark" even, all of these are sub-500ms key sequences that would all take longer than a second if I had to use a mouse. It's not even close.
Part of the problem with the study is that it was comparing using shortcuts vs. mouse on the Mac, where shortcuts are far less useful than they are on Windows. Navigating through menus by keyboard is easier for me with the keyboard on Windows as well; that's not even an option on Mac.
No doubt people can misjudge things. With health changes, I think it is often easier to see changes in the negative direction, like the author when he stopped the soylent trial.
If the changes are great enough in magnitude to be obvious, then you know you're on to something. Also, measures like weight and computer mental agility tests can be quite reliable if you are consistent in your testing, and they aren't subject to misinterpretation or wishful thinking.
Edit:
Below, skore is skeptical that a person can notice a significant change to their own health. I respectfully disagree.
Check out the blog of Seth Roberts. He is a professor who is involved in the quantified self movement. He records many subjective measures qualitatively, and feels he has made some significant discoveries doing so. I won't try to summarize anything here, but anyone interested can read for themselves on his blog. Roberts studies rats, and he is the ultimate human rat.
No, no you don't. That's the whole point of doing scientific experiments. Your brain is very, very, extremely bad at judging objectively. Any subjective "test" is simply no test at all.
> Also, numbers like weight and computer mental agility tests also can be quite reliable if you are consistent in your testing...
That's a pretty big 'if', plus a 'can' and a 'quite'.
> ...and they aren't subject to misinterpretation or wishful thinking
That's the whole point of the OT in this discussion thread. They very much clearly are subject to misinterpretation. A zillion factors can impact weight and "mental agility".
People aren't lab rats because they don't live in labs and aren't rats.
> Below, skore is skeptical that a person can notice a
significant change to their own health.
Yes, a point which still stands.
> Roberts studies rats, and he is the ultimate human rat.
No, Robert might be the ultimate rat conducting experiments on himself. He cannot be his own lab rat, he can only be running his own asylum.
> He records many subjective measures qualitatively, and feels he has made some significant discoveries doing so.
That sums it up perfectly. I suppose all this boils down to is whether you want your science science-y or feel-y. I prefer the science science. Because while I trust them in many things, I don't trust human emotions when making important decisions that are supposed to influence millions over millions of other humans.
I don't know Seth Roberts and I hope I don't come across as overly disrespectful. I'm not saying that it's impossible to make scientific discoveries this way. The problem is that any discovery you make is N=1 and as far away from a double-blind trial as it can be. So this might be an important tool to pre-select what you want to concentrate your actual science on.
But as with the Soylent stuff, I would never in a million years base my own health decisions on some quantified self guy on the internet.
'and they aren't subject to misinterpretation or wishful thinking.' ... right, and we have no way of knowing whether they are.
This is exactly why we have science: people have prior beliefs that affect how they perceive the thing they're trying to measure. People cannot objectively judge their own health in response to a variable. There are thousands of placebo studies that demonstrate this.
I feel you miss the whole point of self experimentation. If your aim is for improved mental function, more energy, less health problems, etc., you don't need to know how to interpret any obscure tests. You'll know.
A low-risk food product (such as this) can simply be trialed in a n=1 trial. If the results are positive, great. If not, go back to what you were doing before.
All the studies in the world only suggest how something might work for you. It is only the n=1 test that matters.