Duplicate notes
I'll first clarify the simplest bug. Starting February 7th, users started to notice duplicate "Audio memos" or "Links" notes:
Spontaneous duplication can cause problems because it affects backlinks. If you open one duplicate, you'll see some backlinks, but if you open another, you'll see different ones.
In addition to system notes like "Audio backlinks", this issue also affected notes imported from external services, for example tags synchronized from Readwise.
Interestingly, the issue occurred right after we rolled out a major rewrite to our database layer. (We are rewriting parts of our database to make it even faster.)
This rewrite touched almost every part of our codebase, so we assumed the bug must be related to the work we just shipped.
We started looking for the problem, but through the lens of our recent release. This was a mistake as it turns out. The error was completely unrelated to our recently published work.
Instead, it was caused by a small improvement related to note aliases that we shipped around the same time. We were trying to improve searching by a note alias, but in doing so, we inadvertently introduced a bug. This caused system notes like Audio Memos to duplicate.
Merge your duplicates
We suspect most of our users haven't even noticed the duplicates. We'd like to ask you to search for your "Audio memos" and "Links" notes, and check for any duplicates.
Luckily, merging is easy. Open one of the copies and place your cursor in the subject line. A menu offering to merge should appear. Confirm, and you're done.
Rewriting the content
The second bug also has to do with duplication, but in a different way. You could encounter it if you followed these steps:
- Create a note, write some content in it
- Create another note, write some content there
- Text from the previous note would end up in the current note
Just from glancing at the description, you can tell this is a serious bug. Text jumping from one note to another is the last thing you want from your note-taking app.
Once again, we started looking through our database rewrite, and in this case it was related.
However, not directly. As a part of the rewrite, we improved part of the codebase that applies changes from database to the editor. We made a small, one-line mistake in this code, and it caused this serious bug.
In both cases, I want to thank users hanging out in our Discord. They provided us with step by step reproduction guides. It made the bug fixing quite easy.
How we'll improve
Our duplication bug, at its essence, is a migration problem. In previous versions of Reflect, our database didn't contain a list of aliases, and then we added it. However, after migrating, the list of aliases was empty for each note. Thus search by alias stopped working.
A simple thing we're doing to improve is adding a checklist. It may sound too simple, but people in critical roles such as pilots rely on checklists every day.
Here you can see some of the initial items we added. One of them is called 'migration from the previous version'.
An engineer will switch back to the previous version, and reset the database. Then they'll upgrade to the latest. They can check the task off after they've verified nothing broke in the process.
For the second bug, we don't have a simple checklist item.
However, from the beginning, the second bug touched critical parts of our codebase. While the code was reviewed, the bug wasn’t caught until it was out in the wild.
Clearly there’s something broken with our process here. We need to clearly distinguish critical PRs from regular ones. They should stand out, they should be written in red and underscored.
We're still hashing this out, but here are some ideas we're considering:
- Mark the PR with "critical:" prefix
- Require at least 2 code reviews
- Require the engineer to record a Loom where they explain the critical change
- Spend more time with in-house testing
We'd like to sincerely apologize to all of our affected users. Please rest assured that reliability is still our top priority.
Written by
.jpg?table=block&id=feb4413a-201a-4159-9826-aff40468e916&cache=v2)