Can I Expense It? There’s an app for that

Lily Smith
27 February, 2020 · 6 min read

Business expenses – what a minefield. If only there was a quick, easy and ideally fun way to check if something is expensable before you buy it. Thanks to the boffins behind the scenes at ANNA, now there is!
Introducing Can I Expense It? The world’s first business expense detection app.

What’s Can I Expense It?

Can I Expense It? uses your camera viewfinder to recognise the objects all around you. Click on an object you want to expense, and the app will let you know immediately if you can claim expenses. Perfect for a spot of on-the-go accountancy advice.

Why did we build a business expense detection app?

One of the questions most commonly asked by self employed people is ‘Can I expense this?’ But what if you could identify expensable items yourself, via your phone? A business expense detection app could end up saving your business time and money. Cha-ching! We thought the idea had legs, so we gathered some of ANNA’s most savvy data scientists and creative… creatives to build an on-the-go method for identifying everyday expenses. And because we’re ANNA, we had a little fun with it.

We were inspired by Google’s Emoji Scavenger Hunt, an object recognition game which uses your phone camera to detect real life objects. Players have to match the emoji they see on screen to a real life equivalent; so if they get a ☔ they need to find an umbrella ASAP. This game got us thinking – could we harness the power of object recognition and machine learning for small business purposes? That way, we could teach our phones to recognise the everyday business expenses all around us.

How it works

Our Creative Technologist Rob started researching object recognition, and found an article about TensorFlow.js – a library of computer code developed by Google which allows us to run real time object recognition from a web browser. Confused? This just means TensorFlow.js was developed to work in a web browser, so you don’t have to download an app to get started. TensorFlow.js helps recognise objects as they appear in your laptop or phone’s camera viewfinder. As you move the camera around, it continues to recognise objects (that’s the real time bit). Thanks to TensorFlow.js, your device has access to a library of images which it can now match up to objects it sees through your camera. You don’t need to tell your computer or phone what it’s looking at. It can (usually) work it out on its own.

Crunching the data science

Using TensorFlow.js we then created a prototype Can I Expense It? that could recognise 90 objects using the COCO-SSD model (that’s a confusing name for some machine learning code that detects real life objects). It worked okay, but most of the 90 objects that this code can recognise – like zebras or frisbees – aren’t really relevant for most businesses.

At this point, our Data Scientist Andrei found another model that could recognise 600 objects, giving us a much better chance of recognising the valid business expenses all around us. Andrei had to convert this model into a format that we could use in the browser with Tensorflow.js. Here we hit another bump in the road: our converted model was too big to load in a web browser. Luckily, Andrei was able to apply a bit of data science wizardry called quantisation, which trades some recognition precision for a smaller model size. We were able to reduce the size of the model by a factor of 4, while maintaining a good standard of browser-based object recognition (ie not too many cups of coffee get mistaken for potted plants). 

It was important for us to host this whole experiment on a web browser, so that it’s easy to share with friends and colleagues. Can I Expense It? runs on both mobile and desktop browsers, but it works best on your mobile, so you don’t have to wave your whole computer around when you’re detecting objects.

Minding our language

Now we had a pretty smart bit of object recognition code working in the background, it was time to package up Can I Expense It? with a good-looking, easy to use interface – and add some all-important expenses advice. ANNA’s Creative Director Will Dogg built a nice, straightforward user flow and then worked his design magic on the landing pages, so that new users could get what the app was all about and start having fun with it immediately. Our copywriter Lily then took the whopping list of 600 objects that our code could recognise, and filtered out all the non-expensable ones (shout out to baseball gloves, martini glasses and picnic baskets!) 

We also made the decision to filter out all the animals from the object detection code, just to avoid any confusion or accidental offence. We all know how upset Shetland ponies can get when they’re mistaken for pygmy goats, right? And finally, we hacked the code so that the app thinks every human face or selfie is a penguin – because no one can get offended by a penguin. Unless you’re a fish.

With some help from HMRC’s website and ANNA’s accountant Laura, we wrote up some handy business expense advice for every expensable object that our app could detect – and a few Easter eggs along the way, too. Although Can I Expense It? was made for fun, the expenses advice we’ve included in the app is all 100% factually accurate. That said, when you use the app Disclaimer Duck is there to remind you to check with your accountant before filing your business expenses.

What we’d do to improve: Can I Expense It? 2.0

We created Can I Expense It? because we believe expenses don’t have to be boring – they’re all about saving your business money, after all. We hope you have as much fun detecting your expenses as we did experimenting behind the scenes at ANNA. Because Can I Expense It? is an experiment, it doesn’t work quite as well as your ANNA app. Building an app for a web browser has its drawbacks, for example some of our code isn’t supported on Chrome on iOS, so Can I Expense It? can’t access the camera on that particular browser. Our object recognition code needs a fairly powerful graphics processing unit, so if you have an earlier device it might struggle to recognise some images. 

When we create Can I Expense It? 2.0 we’ll explore machine learning further, and train our own model to detect more specific objects, rather than relying on a general library of 600 objects that we got out of the box. And we might do something else fun with penguins. 🐧 

We’d love to hear about your experiences with Can I Expense It? – share your photos on our Facebook, Instagram and Twitter and don’t forget to tell us what you’d like to see next from ANNA Money. We’re always listening.

About the author

Lily Smith is a National Film and Television School trained writer, currently freelancing at ANNA Money. She’s collaborated with remarkable, inspiring people and together they've won Gold Lions, D&ADs, and secured $9M in Google Ventures funding.

Read more of Lily's writing
Sort your expenses in the tap of an app
Sure, Can I Expense It? can detect your business expenses, but then what? How about a receipt scanning app that sorts every receipt to an expense category for you.
Enter your mobile number
Save your receipts securely – just snap a picture of your receipt
Leave the organisation and expense categorisation to ANNA
Relax about your tax return – every receipt is matched to a transaction thanks to automated bookkeeping