Bots do the forms. People make the judgment calls. I build the system in between.

The problem

Our teams submit to dozens of external portals, and no two of them behave alike. Fields, page flows, upload rules and verification steps all change from site to site. If I'd written one bot per portal, the codebase would've grown with every new site and broken whenever one got redesigned.

Calls I made

1. New portals come in as configuration.A new portal is added as config. Coverage grew to 71 portals while the codebase stayed the same size.
2. I designed the human steps in from day one.OTPs and CAPTCHAs go to a person, and the bot picks up again once they're done. Trying to automate those steps would have made the whole thing fragile.
3. I defined success so it can't flatter us.Success rate is successful jobs divided by successful plus failed. If a user stops a job on purpose, it doesn't count against the platform.

The approach

In both validators AI does the first pass and a person owns the decision. The Subject Validator summarises and standardises catalogue content. Editors then accept, reject or override each suggestion in a review UI, and every change lands in an audit trail. The Fees Validator routes by confidence. Anything above the threshold is approved automatically and the rest goes to a reviewer.

I tuned the summarisation and set the thresholds myself. That's where most of the real work was. Set them too strict and nothing gets automated. Too loose and bad data goes live.

The problem

To collect companies' official social-media handles, researchers had to visit websites one at a time. Across S&P's company universe that meant millions of visits.

What I built

I wrote a Python and Selenium crawler that ran across 3.4M+ company websites. Domain resolution made sure it hit the right site, and de-duplication left each company with one clean answer. It was the biggest of my S&P automations, which together came to about 168 FTE of capacity.