No Products in the Cart
TL;DR. ~2% of Raise users reported experiencing keyboard freezes where no key input would register while the LEDs remained on. If you are one of those who experienced the freezing issue, download the latest Bazecor and update your firmware to the latest version to turn the freezes into ~3-second resets. Read the post to learn about the problem and how we will fix it long term.
A couple of weeks back, some Dygmates rightfully complained about the freezing issue.
We listened to your feedback and re-evaluated how we were dealing with the freezing issue. This post (and short-term fix) is the result of it.
Usually, companies don't talk openly about the issues they face. But you know, we're not like most companies. We believe in open-development and learning from the community. We believe that any problem is temporary and everything will be solved with enough effort and work.
Our mission is to make the best possible keyboard ever. With your constant help and feedback, we will achieve this together.
The first time we got contacted by a customer about a Raise freezing was in March, right after we released the Bazecor 0.2 update.
Thinking it was related to our latest update, we spent over two months working with a Kaleidoscope firmware developer. After improving the Raise firmware during those months, he told us that he couldn't reproduce the freeze anymore and that he believed it was fixed.
Although the number of freeze cases dropped, occasionally a keyboard would still freeze. So it wasn't fixed.
We started considering other potential causes.
Some clients that bought the first batch of Raise keyboards (up to March 2020) experienced keyboard disconnections related to cables. The cables had faulty soldering, so we changed the supplier and improved the cable quality.
Changing the faulty cables fixed many freezing cases, but again, it didn't fix 100% of them.
We picked up keyboards that experienced freezing and tested them. In most cases, the keyboards we picked up from customers wouldn't freeze.
This made us think that it could be related to their setup around the Raise.
In some cases, we were able to fix the freezes by uninstalling drivers of other peripherals. In other cases, by using a different USB port or by updating or downgrading the firmware. And for others, by removing a device such as a KVM switch.
In no case were we able to replicate it consistently ourselves.
This is the data we gathered from talking to customers:
From those Dygmates:
We were able to fix most keyboards that experienced freezes except a handful where the cause was related to their environment.
Even if we are able to fix most cases, this situation is still not ideal. It's annoying for the customer and it requires a lot of work from our customer support team. Plus, we were paying the extra cost of shipping replacement components around the world.
We needed to find a solution ASAP.
After analyzing deeply what was happening, having many internal meetings and bumping our head against the wall a few times, we reached a few conclusions:
*The I2C is the communication protocol used so both sides speak to the Neuron. But I2C was designed so chips talked to each other in the same PCB. The Raise has 5 PCBs, 3 chips in 3 different PCBs and they are connected by two different cables.
The I2C protocol wasn't created for this kind of "electrical stress" environment.
Now that we know what's going on, we can focus on fixing it. Hurraaay!!🎉
Now that we know that the main issue is the I2C, we can focus all our efforts on "strengthening" it. This can be done short term via firmware and long term via hardware. We can even solve the issues provoked by external devices.
We have a 4-step masterplan to fix the freezing.
This is a short term band-aid, but the experience for any user is much better than having to unplug and replug the Neuron.
What we are doing here is telling the keyboard to completely reset when the communications in the I2C channel gets blocked.
It's kind of an overkill solution because we aren't just resetting the I2C channel, but the whole keyboard.
The next step addresses this.
This sounds a bit better.
Instead of resetting the whole keyboard, we will refresh the side of the keyboard that has the I2C channel issue.
This would be close to instant and in most cases even imperceptible by the user.
Now we're talking!
This would make any I2C refresh unnoticeable by any user.
The Raise passed all electronic certifications (ESD, EMI, and EFT), so it already has the required electronics to be safe and stable.
But there are some scenarios that we didn't consider when designing our PCBs, thus have made us want to reinforce the design:
With this in mind, we can improve the sturdiness of our electronics by:
We are on Step 1. You can now download the latest Bazecor version and update your keyboard to the latest firmware that turns the freezes into resets.
Our developer, Alex, has already started working on the Step 2, turning the freezes into instant refreshes.
On top of that we are looking for a firmware developer to help us with this. If you know of any, please email us at firstname.lastname@example.org.
And at the same time, our Product Manager Manel is working on the Step 4. He's looking for an electronic engineer with experience in consumer electronics or medical devices and specializes in emissions. If you know of any electronic engineer that fits the criteria, please tell them to contact us at email@example.com or let us know.
We hope we can have Step 2 by February. We still don't know exactly when to expect Step 3. We will learn more about this once we've investigated more into the previous step.
Finally, we plan to do Step 4 for the next mass production batch of Raise keyboards, which will be by mid-2021. We will keep you posted on our developments!
A very special thanks to Babak Javadi!
He reached out to us when he experienced the freezing issue. He went out of his way to do experiments to find where the problem was coming from. We had a video call with him and he explained to us how he was able to replicate the freezing issue and offered his solutions based on his expertise.
It's so amazing to have great Dygmates like him! Thank you Babak!
Please check our Reddit thread to join the conversation about this topic and to provide feedback and suggestions! 😊