ClearFracture.ai is a software startup building an AI agent platform focused on data curation, data engineering assistance, and knowledge preparation for enterprise AI use cases. The company’s broader view is that enterprise AI systems need well-prepared data and structured knowledge representations in order to reason effectively, take useful action, and support complex business or technical workflows. ClearFracture is not looking to provide data to students. Instead, the company is interested in a project where students can analyze the performance of AI agents themselves, using the telemetry and reasoning traces generated as the agents operate.

This project is designed for the USC Upstate Graduate Business Analytics Enterprise Project Sequence because it gives students a rich, repeatable, self-contained analytics environment. Rather than asking ClearFracture to hand over proprietary data, students would run an AI agent against selected open-source code repositories or controlled test cases. The agent would produce operational data, including reasoning traces, processing steps, token usage, intermediate outputs, timing, success or failure states, and other telemetry. The recommended agent produces this telemetry automatically and compatible for analysis using LangFuse or LangSmith. Students would design experiments, then analyze that data to understand how the agent behaves, where it performs efficiently, where it struggles, and how its configuration or control logic could be improved.

Over the course of the analytics sequence, students will treat AI agent evaluation as a business analytics and data science problem. They will define performance metrics, collect agent telemetry, clean and structure trace data, visualize patterns, identify failure modes, compare runs across scenarios, and recommend optimization strategies. The goal is to help ClearFracture better understand how its agent behaves under different conditions and how telemetry analysis can create a feedback loop for improving agent accuracy, reliability, speed, and efficiency.

The project should focus on practical analytics outputs that are valuable to both ClearFracture and USC Upstate students. Students may analyze how different inputs, repositories, task types, prompt strategies, model settings, or agent workflows affect performance. They may examine where the agent uses excessive tokens, where it takes unnecessary processing steps, where it produces incomplete outputs, or where specific types of repositories create recurring challenges. The students’ work can help ClearFracture develop a more systematic approach to measuring and improving agent performance without exposing any sensitive customer data.

The final deliverable will include a telemetry data model, performance measurement framework, exploratory dashboards, agent behavior analysis, optimization recommendations, and a final presentation explaining how ClearFracture can continue using telemetry analytics to improve agent behavior over time. If the project progresses successfully across multiple courses, students may also prototype a repeatable evaluation workflow or dashboard that ClearFracture can use when testing future versions of its agent.