Devlin Peck
The Rise of the Learning Engineer
A New Role Between Instructional Design & Software Engineering
The Workshop
- In March, I hosted a 90-minute workshop: teach IDs to build AI roleplays in Storyline
- We covered servers, API keys, coding agents, etc.
- There were unexpected roadblocks
- Some got there. Many didn't.
What Would You Build Next?
- 1TRAIN: A full tutorial series
- 2SUPPORT: Job aids and setup office hours
- 3AUTOMATE: A tool that handles the technical side for them
Remove the Need for Learning
- 3AUTOMATE: A tool that handles the technical side for them
- Describe the character for the role-play
- AI generates the simulation
- Copy the link
- Paste it into Storyline
- No servers. No API keys.
Job #1: Make the learning unnecessary.
The learning engineer's first job is to eliminate the need for learning. They build software and own its impact on human performance.
How the Work Is Shifting
Clicking
- You build every screen by hand
- Storyline, click by click
- Your output is capped by your hours
Delegating
- You describe the outcome to AI agents
- You give them tools, context, and checks
- They build. You review and decide.
Toil
- Toil is the repetitive manual work that you do, over and over
- Google's rule (Vivek Rau, Site Reliability Engineering): engineers spend no more than 50% of their time on toil
- The other 50%: building systems that take the toil away
- Today's real work becomes tomorrow's toil
The Team That Automates Toil
- An L&D team was spending 70% of their time building and updating software sims in Articulate Storyline
- The software changes every few weeks
- What I proposed: a system with agents that check each release overnight
- It drafts the updated sims for review in the morning
The Learning Engineering Loop
Analyzethe business goals and needs
→
Automatewhat doesn't need a person
→
Embed helpinside the work
→
Design practicefor the skills people need
→
Measurethen improve
↻ Measure → Analyze
Start With a Business Number
- Faster ramp-up for new hires
- Fewer expert hours spent training
- Revenue-generating people back on revenue work
- On-the-job KPIs: customer satisfaction, handle time, errors
Eight Doctor-Hours Per New Hire
- Telehealth onboarding
- Senior doctors run about 8 hours of 1:1 practice calls per new hire
- The ask: add AI practice
- The real goal: get doctors back to patients
What Would You Try First?
- 1TRAIN: AI practice calls instead of doctor role-plays
- 2SUPPORT: AI help during live calls
- 3AUTOMATE: Hand the routine calls to AI
What We Actually Built
- I built: a practice copy of their real software
- AI patients call in, AI scores the call and gives feedback
- Senior doctors review instead of running every call
- Their software team built: AI assist inside the live software
- Our choice: no AI assist in practice
The Dial Moves as Trust Grows
- Before: senior doctors ran every practice call
- Now: AI runs some practice calls, doctors review
- Next: AI assist on live calls, checked less as it proves itself
- Someday: AI takes routine calls, doctors take escalations
- Each notch is earned with evidence and can be rolled back
Who Handles the Exceptions?
- AI takes more of the routine work
- People get the hard, weird cases
- But the routine work was how they practiced
AI Assists on Most Calls. Where Do You Invest Next?
- 1TRAIN: Rehearse the hard calls
- 2SUPPORT: An AI coach that guides the doctor through hard calls
- 3AUTOMATE: Improve the system so AI can take hard calls, too
Practice the Hard Calls Without the Assist
- 1TRAIN: Rehearse the hard calls
- In the practice copy of their real software
- AI patients with messy, missing info
- No AI assist, so doctors can take over when it fails
- Scored on a full rubric, with feedback and another try
- Skilled doctors catch the AI's mistakes instead of trusting it blindly
How Do We Know It Helped?
- We're evaluating:
- Doctor-hours per new hire
- Key on-the-job metrics at 7, 30, and 90 days
- Time to readiness for real calls
- Tie the project to KPIs, not just surveys
Someone's Skill Is Always the Backstop
- When the AI is wrong, the operator has to catch it
- When the operator misses it, it's on the system's designer
- Automation doesn't remove human error. It moves it.
- Strongest setup: the people who do the work help design the system
Then I Did It to My Own Work
- First: building in Storyline, click by click
- Then: AI agents did the building, and I reviewed every change
- Soon, reviewing took almost all of my time
- Today's real work had become tomorrow's toil
Review Became My Bottleneck
- What would you do in this situation?
- I built a system that conducts reviews for me
- Checks every screen with a screen reader
- Analyzes new code for potential security risks and bugs
- Conducts real voice sessions to ensure no regressions
- Reviews breakpoints across screen sizes and browsers
I Only Get Pulled In When It Matters
- My test agents get a real browser, a screen reader, AI voices, a test database, and screenshots
- They fix and re-test until every check passes, sometimes for hours
- Nothing goes live until every check is green
- What reaches me: design changes, big decisions, what a test can't judge
- The checks keep improving, so I get pulled in less
I Rebuilt My Workspace, Too
- Started: coding agents in the terminal and VS Code
- Then: a terminal multiplexer to run many agents at once
- Then: I built Peck so my team could use agents without the command line
- Drop in files, run agents side by side, check in from my phone
My Workspace
Screenshot of Peck with several agents running at once in the sidebar
"Isn't This Just ID Rebranded?"
- Partly, yes. Gilbert, 1978: fix the environment before training the person
- Learning engineering extends it: remove the need for learning first
- What's new: you can build the fix yourself, with AI agents
- Honestly, the title doesn't matter
- People who can work between ID and software engineering will be very valuable
Your First Project Is Your Own System
- Start with your own toil (or your team's)
- Low risk, and it adds value right away
- It's where your skills get proven
- Then bring it to your clients, internal or external
- How I started: my own business first, until I knew where it breaks
Start Small
- One task you repeat every week
- Define what success looks like
- Test it before anyone else uses it
- Later: a full job-task analysis to find what to automate next
What to Learn First
- A coding agent: Claude Code (or Codex)
- Describe the outcome; give it the right tools and context
- MCP connects it to your other tools
- Git and GitHub: version history, branches, previews, teamwork
- Tests: make the agent prove it works before anything goes live
- Then: run several agents at once, across projects
Readings
- Vivek Rau — Eliminating Toil
- Lisanne Bainbridge — Ironies of Automation
- Thomas Gilbert — Human Competence
- Anthropic — Building Effective Agents
What Should People Still Be Able to Do?
- Cut the learning nobody needs
- Make the job easier to do right
- Keep people sharp enough to catch the AI when it's wrong
Where do your experts keep getting pulled in?
Book a free call: a consult or just a coffee chat. Let's compare notes.
devlinpeck.com/devlearn
devlinpeck.com/devlearn
Questions
Your workflows · Your objections · Your first project