← Back to selected work

Simplifying ML experimentation.

Turning repeated ML configuration into a self-service workflow for forecasting scientists.

My roleSoftware Engineer
UsersForecasting scientists
FocusML platform & model lifecycle

PRODUCT MISSION

Help scientists spend more time improving forecasts and less time setting up the work.

Problem & goal

Forecasting scientists need to test ideas repeatedly: train a model, fine-tune it, generate predictions, and compare them with historical demand. Before that learning could begin, they had to configure and orchestrate the experiment.

That repeated setup was a workflow problem. Making one job easier to launch would still leave scientists looking for its status and outputs.

The goal: give scientists a consistent way to request an experiment and follow it through to usable results.

Solution

I designed and drove delivery of a self-service experimentation platform covering training, fine-tuning, inference, and backtesting. Scientists could request work through natural language, with agent tools calling the underlying APIs and orchestrated workflows.

The experience connected three steps: supply the task and inputs, follow the run, and retrieve the outputs. The model computation continued asynchronously after submission.

User journey

The scientist’s experiment journeyA forecasting scientist chooses a training, fine-tuning, inference, or backtesting task, supplies its inputs, launches a workflow, and tracks its status. Completed runs expose results; a failed run can be inspected before the next attempt.The forecasting scientistGoal: run an experiment and find its results.1. Describe the taskChoose model + inputs2. Launch the runWorkflow orchestrates jobs3. Check progressTrack the run statusComplete4. Review resultsFind experiment outputsFailedInspect the runUse status to investigateNext attemptThe scientist’s experiment journeyA forecasting scientist chooses a training, fine-tuning, inference, or backtesting task, supplies its inputs, launches a workflow, and tracks its status. Completed runs expose results; a failed run can be inspected before the next attempt.Forecasting scientistRun and review an experiment1. Describe the taskChoose model + inputs2. Launch the runWorkflow orchestrates jobs3. Check progressTrack the run status4. Review resultsFind experiment outputsCompleteIf a run fails:Inspect status before the next attempt.
Simplified task flow. Setup, execution status, and result retrieval are part of the same experience.

My role

I connected the scientist’s task to the platform implementation, from workflow design and agent tools to configuration, orchestration, status checks, and result retrieval. I also mentored an intern through implementation.

I kept the user’s full task in view while working through the constraints of the existing ML platform.

Decisions

Make the full task self-service.

Launching a job is only the start. I included status checks and output retrieval so scientists could follow their work without treating the platform as a black box.

Trade-off: this meant a broader integration than a launch-only interface, but it addressed the points where users would otherwise need more help.

Put a simpler entry point on repeatable workflows.

Natural language helped scientists express the task. APIs and orchestrated jobs provided a defined execution path for training, inference, and evaluation. The interface and execution layer had different responsibilities.

Trade-off: this required explicit tool contracts and supported inputs. It also kept the execution path inspectable when a request failed.

Measure the work we actually removed.

I separated preparation and launch time from model compute time. The platform reduced repeated setup; it did not make every training run finish faster. Daily job volume showed use of the workflow, while repeat use by individual scientists would require a separate measure.

Related delivery: making model validation easier to submit

Before a forecasting model could move toward production approval, scientists needed to run a supplied validation workflow and make its outputs available to an approver. I automated creation and submission and connected the results back to the validation record.

The workflow library was too large for the existing API’s deployment package and had its own Python compatibility requirements. I isolated it in a container-based submission function, then reused the existing job-monitoring path to return status and output locations across AWS accounts.

The trade-off: an additional service boundary to maintain in exchange for separating the API from heavyweight execution dependencies. A completed workflow supplied evidence for the approver; it did not automatically approve the model.

Scientists could create a validation in under three minutes. This was a separate validation-submission outcome, not the runtime of the ML computation.

Impact

~90%Less experiment setup time. The reported improvement covers preparation and launch, with model compute time measured separately.
20–25Experiment jobs per day supported by the workflow. Job volume shows platform usage, rather than the number of unique scientists.

Scientists could launch supported experiment types and find status and results through a connected workflow.

Reported delivery outcomes from my project record. A comparable-cohort breakdown and measurement window are not included in this case study.

What I would measure next

Primary measure: the share of experiment requests that reach accessible results without engineering help.

Supporting measures: setup time by experiment type, weekly active scientists, and repeat use. Guardrails: failed runs, unsuccessful result retrieval, and compute cost per comparable workload.

These are proposed follow-up measures, not additional historical results.

What I took away

An internal platform earns adoption by removing the work around the task. The launch, the wait, and the result all need to make sense to the person using it.