Lessons from Ki

Encryption is magic. On one side of the world you scramble a message, send it over any number of unsecured wires, allow literally anybody to view it, and unscramble it on the other side knowing that only you and the sender could see it. Encryption is vital to the internet, privacy, security, anonymity and the modern world.

Encrypting files on your computer is easy enough but, for me, encrypting files on a phone proved more difficult. I’d searched high and low for a good encryption Android app, but to no avail. Most apps I found in that category weren’t well designed, were too bloated with features I didn’t want and some didn’t even encrypt anything at all – the ‘security’ was ridiculously easy to bypass.

The worst example I found was an app called AppLock. To sidestep the security, all you had to do was clear the app’s data from Settings, re-open the app to setup a first-time password, login with that new password and Bam! Three easy steps and you’re in, with full access to absolutely everything. This app, by the way, is still available on the Google Play Store and has over 250,000 downloads.

Being a fairly decent Android developer myself, I decided to write my own version. I’d design it to be as simple and secure as possible. No easy bypasses, no obnoxious UI design, no featuritis and absolutely everything would be encrypted using the user’s own password. Just a simple and secure encryption app, that anyone could use and download for free. I called my app ‘Ki’, after the magic energy from Dungeons and Dragons.

I learned a lot while writing Ki, not just about Android development, but also about design, architecture, marketing and product launch. This article is the collection of lessons that I learned while building Ki, from prototyping to launch.

Ki is open source, and can be found on GitLab at https://gitlab.com/midfords/Ki. The product website is hosted here, and the app can be downloaded here.

Design Evolution

My first crack at this project looked completely different from the app that eventually launched. The UI was quite different, the storage and encryption implementations were crude and even the title and logo changed. The accumulation of small bugs and design oversights, over time, warranted a complete redesign of the entire project.

The first version was called uCrypt. It’s almost inappropriate to compare uCrypt to Ki, since the two apps differ in every way except objective. I’d estimate less than half the original code was kept and, while the former app was written in Java, the latter was largely written in Kotlin. The encryption methods in the related apps also evolved. uCrypt had crude implementations of a key-gen method, and files had to be entirely read into memory before being encrypted. This caused some nasty bugs that were difficult to fix and potentially security issues as well.

The original also didn’t use a proper database, and instead stored everything in encrypted text files on the file system. Each file would store its metadata in a JSON text file and, on startup, the complete contents of every metadata file would be read into memory and decrypted. Only then could the app start up and generate the list items for the view.

While developing uCrypt, most of its UI was improvised as needed. As a result, the app interface became quite unintuitive. When revisiting the app myself after a short break, I would be momentarily confused about how to use and navigate the app. As I redesigned the app, I took time to make a list of features and iterated through various UI mockups until I knew exactly what I would be making. This proved enormously helpful.

This category of apps is in a bit of a difficult position. It encrypts and manages peoples’ data in such a way that, if there were ever any programming errors, you could lose that data forever. User trust would be irretrievably lost. This could happen because of a bug, future update or database change that fundamentally changed the method of encryption or key generation. Keeping this in mind, when designing the database and encryption methods, I was careful to make each part of the program future-proof. This way the app could be expanded on in the future without worrying about losing backward compatibility (and everybody losing access to their data).

List Item Observer Chain

Ki is mostly written in Kotlin, and uses many of the Android Jetpack Architecture Components (released in 2018). The combination of LiveData, ViewModel, Room and Kotlin’s coroutines were especially useful in making the app responsive, efficient, and uncomplicated.

The ViewModel and LiveData components allow for complete decoupling of the view and model, and allows all long running operations to be done asynchronously. By combining LiveData objects into, what I call, a “List Item Observer Chain”, we can simplify the View-ViewModel interface enormously. Only two entry points and two exit points are exposed to the view. The entry points are Search Query and Current Folder Id. The exit points are Current Folder Name and Results. All the internal plumbing between the two entry points and the results point is completely hidden from the implementing view. This moves all logic out of the view and keeps the view as simple as possible, just set a maximum of two values and display the list of results.

Diagram of the List Item Observer Chain in Ki.

This observation chain will also respond to any asynchronous changes to the database tables automatically. If the database changes in the background (perhaps, because a large file has just finished importing), it will trigger a new set of results, and the view will update without a refresh.

There is one other benefit that this design offers: only the list items that are currently being observed will be decrypted. When the file/folder entities are mapped onto list items, the view properties are not decrypted until the getter method is called, not when the list item is mapped. This effect prevents most items from being stored in memory in plain text. So, for example, if you have a list of 100 items, and your phone’s display is 6″ wide, then only the items that are currently being viewed will be decrypted at any given moment.

Writing a File System

As I mentioned before, the original uCrypt stored all metadata as individual text files. This scheme caused a number of issues, such as the poor efficiency of launching the app and the security of having every item stored in memory in plain text. One of the stranger issues this caused was the app not being able to save empty folders.

Each metadata file looked something like this:

This JSON describes a file called “my_file.txt”, encrypted using AES and stored in the path “/Documents/My Files/my_file.txt”. Can you see the problem? Folder names were stored in a file’s metadata only, so it was impossible to save a folder path without an associated file keeping track of the “path” list. uCrypt had no way of storing those folder names outside of the file metadata. Thus, no empty folders allowed.

The move action also proved challenging under this design. Moving a folder required updating the metadata of all the files inside that folder and every subfolder, then rewriting each result back to the file system. This was terribly inefficient and difficult to implement. The same problem occurred when renaming a folder, every file would need to be checked for that folder and have its “path” property updated and saved.

In version 2.0, I dispatched with the file system and used a proper SQL database instead. The database was fairly simple to write and accounted for all the problems previously mentioned.

Now, instead of a list of folder names, the path reflects the actual model of a file system. Each file and folder has a pointer to its parent folder. This is done by referencing another row in the folder table, so the Folders table actually references itself recursively. To indicate that an item is in the root folder, the parent value is set to NULL. This is more efficient for rename, move, get and delete actions on the database. It allows for empty folders as well. In fact, it even allows for multiple folders to have the same name, something a real file system can’t even do.

Now we can fetch the contents of a folder with a simple SQL statement:

And fetching the contents of the root folder with:

The database enforces a constraint on the Folder table, so a folder row can never be deleted if it is being referenced by another File or Folder row. This way, the database can never loose reference to a folder or file. Deleting a folder from the database must be done recursively, deleting each of the folder’s children before it can be deleted itself. Unfortunately, this must be done synchronously, since the next folder can’t be deleted until all its children are known to have been deleted (asynchronous deletes won’t work). Thankfully, Kotlin’s coroutines come to the rescue yet again, and lets Ki perform all these slow recursive operations on a background thread, independent of the UI. As the items are deleted one by one, the List Item Observer Chain automatically updates the view to reflect the changes.

Moving to an SQL database, in combination with the LiveData component, allowed all major operations to be done asynchronously. The difference in responsiveness was night and day.

Featuritis

You can always dream up new features. When I first started programming Ki, my scope for the project was huge, and ended up being far too much to realistically finish. It seemed I was consistently underestimating how long these features would take to develop fully. Towards the end, all those little bugs and fixes I had been putting off came back to haunt me. It felt like half my time was spent just wrapping-up all the little bits I had put off before. After fixing each bug I’d discover two more, and after finishing each feature I’d dream up an idea for yet another.

The scope of the project was never ending. Probably the most important lesson I can pass on to the reader is this: Just call it done. My final product didn’t include absolutely everything I wanted it to, but it included enough to be functional and it actually released.

There were many features that didn’t make it into version 1.0 of Ki. I had planned to let the user generate unique passwords to encrypt individual files or folders, view complex file types in-app (video, document, text, etc.), snap encrypted photos directly in the app and display notifications for long-running background operations (imports, for example).

Planning for some of these features down the road, I made sure to design the database to include “Algorithm” and “Hash” fields, even though there is currently no purpose for them. Remember, if you try to pack absolutely everything into version 1.0, you’ll never end up releasing anything.

Crossing the Finish Line

There are many aspects to launching a product aside from building the actual software. It requires product pages, developer accounts, websites, documentation and marketing materials, just to name a few.

One of the bigger launch-related tasks for Ki was building a product website (to save money, I hosted it from my personal webserver under the subdomain ‘ki’). The website was very simple and I wrote it in plain HTML, CSS and JS. It includes a title page, screenshots, feature descriptions, basic contact information and the ‘Get it on Google Play’ button.

As mentioned before, the name Ki was inspired by the magic energy from Dungeons and Dragons. There were many attempts at drawing a simple logo that didn’t blatantly rip off another product. The final version I arrived at was loosely inspired by the word ‘Ki’ itself.

Ki logo inspiration. The left two rhombi form the ‘K’ and the right two form the ‘i’.

The screenshots were carefully considered to include everything the app offered. I took them using my personal Samsung Galaxy S6. The more complicated images (such as the app logo, icons and banners) were drawn in Gimp.

The last step to launching an app is to submit it to Google’s review process. If nothing goes wrong, your app should be approved in a week or two. From there, it can take another couple of days to actually start appearing in the Play Store search results. At first, I found it difficult to find my app. Even by searching for the exact term “Ki: Keep Files Safe”, my app would be buried far down in the results. Apparently Google’s search algorithm favours the number of downloads an app has, rather than delivering exactly what you search for. Oh well…

Conclusion

The Google Play Store has over 3 million apps (as of February 2020), and that number grows more every year. The odds of your app becoming a smash hit is akin to winning the lottery. For every Angry Birds there’s about a thousand failed apps. Originally, I wanted to charge $2.99 for Ki, but later decided that its function would be better served as an open source project. So, I opened the GitLab repository to the public and set Ki’s price to free.

Building this project taught me a lot about modern Android development. At the time, Kotlin, Android Jetpack and Android’s SQLite were all new to me. I can’t say enough good about the LiveData, ViewModel and Room components. These pieces fit together so well, and make developing clean Android application straight forward and enjoyable.

Though this app wasn’t a commercial success by any definition, I would still consider it a personal success. I put a lot of work into the app’s design and development, and I’m proud of the final result. The knowledge I gained along the way will likely be useful elsewhere, and now I have a great project to talk about during job interviews. My goal was to make a simple and easy-to-use encryption app for Android, and I think Ki fits that description well.

Cheers!   🍻