Nikolas Moatsos: my Internship at ANNA

Trust and collaboration

Over the past three months, I’ve had the opportunity to work as a Data Science Intern in the Business Admin team at ANNA, under the guidance of Nick Turusin and Liev Garcia.

From the very first week I was trusted with real tasks and hands-on involvement in the team’s projects. What struck me most was how quickly I was treated not just as an intern, but as a data scientist in my own right. At ANNA, there’s a strong culture of trusting people, giving them responsibility, and focusing on getting meaningful results. This meant that I had the freedom to work independently – but I could always rely on Nick and Liev for advice and support. This sense of trust and collaboration made my internship really rewarding.

My task: building LLM Supervisors

The main focus of my internship was developing LLM supervisors – tools designed to automatically evaluate AI agents in real time.

These agents are used across ANNA’s many automated, client-facing features, and it’s important to monitor their performance so we know they’re working as they should. Traditionally, this quality assurance process required people to manually review countless conversations, which was time-consuming and repetitive.

By training LLM supervisors on examples where humans had already evaluated dialogue quality, we were able to automate this process, making it faster, more reliable, and scalable. Our work now gives ANNA useful insights into how its AI agents are performing, and also highlights the areas where improvements can be made.

Three LLM supervisors

I worked on three different supervisors, each measuring:

• Contextual consistency – whether an agent’s responses followed a logical flow.
• Goal achievement – whether the agent successfully completed its assigned task.
• Grounding – whether the agent’s answers were based on the given context (rather than inventing information).

I needed a wide range of skills to develop these supervisors. I carried out data annotation to create training and test datasets, used statistical analysis to measure performance, and worked on deploying the supervisors in production.

I also helped design a monitoring dashboard so that colleagues at ANNA could easily track performance and spot opportunities for improvement. Being involved in every stage of this process was challenging and exciting, and gave me a real sense of end-to-end ownership of a project.

Improving the AI Tax Agent

Another key project I contributed to was enhancing ANNA’s existing AI tax agent. My goal was to extend its capabilities so it could help sole traders with their self assessment forms. This meant that customers completing their self assessment through ANNA could ask the agent specific questions and get immediate, personalised responses.

Since the self assessment process can be cumbersome and lengthy, having a chatbot that can answer questions along the way is really helpful and reassuring for customers. It reduces uncertainty and provides guidance at each step of the journey.

I adapted the AI tax agent to support self assessment queries, benchmarked its performance, and delivered a working prototype. This feature is now on track for release and will soon be helping ANNA’s customers. Being part of this project gave me valuable experience in adapting existing AI systems to new, practical use cases.

Collaboration and learning

My work at ANNA was always collaborative, involving close work with software engineers, tax experts, product owners, and other data scientists.

Working with colleagues from such different backgrounds showed me how important communication and teamwork are when building real-world solutions. Along the way, I improved my technical skills, particularly in natural language processing and production-level software engineering, and my soft skills, such as explaining technical results clearly, and discussing requirements with people who approach problems from different perspectives.

A typical day during my internship was very varied: team catch-ups, planning discussions, coding sessions, testing prototypes, and analysing real-world customer data.

This gave me a true sense of what it’s like to work as a data scientist in a fast-paced environment, solving problems end-to-end – and always keeping customers’ needs in mind.