Bounds bounds bounds bounds. The important part for humans seems to be maintaining boundaries for AI. If your well-tested codebase has the tests built thru AI, its probably not going to work.

I think its somewhat telling that they can't share numbers for how they're using it internally. I want to know that Microsoft, the company famous for dog-fooding is using this day in and day out, with success. There's real stuff in there, and my brain has an insanely hard time separating the trillion dollars of hype from the usefulness.

So far, the agent has been used by about 400 GitHub employees in more than 300 our our repositories, and we've merged almost 1,000 pull requests contributed by Copilot.

In the repo where we're building the agent, the agent itself is actually the #5 contributor - so we really are using Copilot coding agent to build Copilot coding agent ;)

(Source: I'm the product lead at GitHub for Copilot coding agent.)

I'd like a breakdown of this phrase, how much human work vs Copilot and in what form, autocomplete vs agent. It's not specified seems more like a marketing trickery than real data

I'm curious to know how many Copilot PRs were not merged and/or required human take-overs.

every bullet hole in that plane is the 1k PRs contributed by copilot. The missing dots, and whole missing planes, are unaccounted for. Ie, "ai ruined my morning"

Everyone who has used AI coding tools interactively or as agents knows they're unpredictably hit or miss. The old, non-agent Copilot has a dashboard that shows org-wide rejection rates for for paying customers. I'm curious to learn what the equivalent rejection-rate for the agent is for the people who make the thing.

How does this align with Microsoft's AI safety principals? What controls are in place to prevent Copilot from deciding that it could be more effective with less limitations?

Most developers don't love writing tests, or updating documentation, or working on tricky dependency updates - and I really think we're heading to a world where AI can take the load of that and free me up to work on the most interesting and complex problems.

What is the job for the developer now? Writing tickets and reviewing low quality PRs? Isn't that the most boring and mundane job in the world?

Those orgs that value high-quality documentation won’t have undocumented codebases to begin with.

And let’s face it, like writing code, writing docs does have a lot of repetitive, boring, boilerplate work, which I bet is exactly why it doesn’t get done. If an LLM is filling out your API schema docs, then you get to spend more time on the stuff that’s actually interesting.

So they won’t like working on their job ?

I use all of these tools, but you also know what "they're doing"...

I know our careers are changing dramatically, or going away (I'm working on a replacement for myself), but I just like listening to all the "what we're doing is really helping you..."

Doing either of them _well_ - the way you do when you actually care about them and they actually matter - is still so far beyond LLMs. Good documentation and good tests are such a differentiator.

Really cool, thanks for sharing! Would you perhaps consider implementing something like these stats that aider keeps on "aider writing itself"? - https://aider.chat/HISTORY.html

where are we wrt the agent surveying open issues (say, via JIRA) and evaluating which ones it would be most effective at handling, and taking them on, ideally with some check-in for conirmation?

Or, contrariwise, from having product management agents which do track and assign work?

The entire website was created by Claude Sonnet through Windsurf Cascade, but with the “Fair Witness” prompt embedded in the global rules.

If you regularly guide the LLM to “consult a user experience designer”, “adopt the multiple perspectives of a marketing agenc”, etc., it will make rather decent suggestions.

I’ve been having pretty good success with this approach, granted mostly at the scale of starting the process with “build me a small educational website to convey this concept”.

Who would've thought (except you) that this would be one of the things that AI would be especially suited for. I don't know what this progression means in the long run. Will good engineers just become 1000x more productive as they manage X number of agents building increasingly complex code (with other agents constantly testing, debugging, refactoring and documenting them) or will we just move to a world where we just have way fewer engineers because there is only a need for so much code.

Well, that's back rationalization. I saw the advances like conducting meta sentiment analysis on medical papers in the 00's. Deep learning was clearly just the beginning. [0]

> Who would've thought (except you)

You're othering me, which is rude, and you're speaking as though you speak for an entire group of people. Seems kind of arrogant.

0. (2014) https://www.ted.com/talks/jeremy_howard_the_wonderful_and_te...

My view is in between yours: A bit of column A and B in the sense both outcomes to an extent will play out. There will be less engineers but not by the factor of productivity (Jevon's paradox will play out but eventually tap out), there will be even more software especially of the low end, and the ones that are there will be expected to be smarter and work harder for the same or less pay grateful they got a job at all. There will be more "precision and rigor", more keeping up required by workers, but less reward for the workers that perform it. In a capitalist economy it won't be seen as a profession to aspire to anymore by most people.

Given most people don't live to work, and use their career to also finance and pursue other life meanings it won't be viable for most people long term especially when other careers give "more bang for buck" w.r.t effort put into them. The uncertainty in the SWE career that most I know are feeling right now means to newcomers I recommend on the balance of risk/reward its better to go another career path especially for juniors who have a longer runway. To be transparent I want to be wrong, but the risk of this is getting higher now everyday.

i.e. AI is a dream for the capital class, and IMO potentially disastrous for social mobility long term.

Meanwhile, engineers are using it for code completion and as a Google search alternative.

I don't see much difference here at all, the only habit to change is learning to trust an AI solution as much as a Stack Overflow answer. Though the benefit of SO is each comment is timestamped and there are alternative takes, corrections, caveats in the comments.

I don't understand what this has to do with AI adoption at MS (and Google/AWS, while we're at it) being management-driven.

This was true up around 15 years ago. Hasn't been the case since.

Similar to google. MS now requires devs to use ai

The actual quote by Satya says, "written by software".

In this context, assuming that humans will still be able to do high level planning anywhere near as well as an AI, say 3-5 years out, is almost ludicrous.

I noticed that LLMs need a very heavy hand in guiding the architecture, otherwise they'll add architectural tech debt. One easy example is that I noticed them breaking abstractions (putting things where they don't belong). Unfortunately, there's not that much self-retrospection on these aspects if you ask about the quality of the code or if there are any better ways of doing it. Of course, if you pick up that something is in the wrong spot and prompt better, they'll pick up on it immediately.

I also ended up blowing through $15 of LLM tokens in a single evening. (Previously, as a heavy LLM user including coding tasks, I was averaging maybe $20 a month.)

This is a feature, not a bug. LLMs are going to be the next "OMG my AWS bill" phenomenon.

Some well-paid developers will excuse this with, "Well if it saved me 5 minutes, it's worth an order of magnitude than 10 cents".

Which is true, however there's a big caveat: Time saved isn't time gained.

You can "Save" 1,000 hours every night, but you don't actuall get those 1,000 hours back.

What do you mean?

If I have some task that requires 1000 hours, and I'm able to shave it down to one hour, then I did just "save" 999 hours -- just in the same way that if something costs $5 and I pay $4, I saved $

You still get your 24 hours, no matter how much time you save.

What actually matters is the value of what is delivered, not how much time it actually saves you. Justifying costs by "time saved" is a good way to eat up your money on time-saving devices.

    Hourly_rate / 12 = 5min_rate

    If light_edit_cost < 5min_rate then savings=true

LLMs are now being positioned as "let them work autonomously in the background" which means no one will be watching the cost in real time.

Perhaps I can set limits on how much money each task is worth, but very few would estimate that properly.

The only people who believe this level of AI marketing are the people who haven't yet used the tools.

> which means no one will be watching the cost in real time.

Maybe some day there's an agentic coding tool that goes off into the weeds and runs for days doing meaningless tasks until someone catches it and does a Ctrl-C, but the tools I've used are more likely to stop short of the goal than to continue crunching indefinitely.

Regardless, it seems like a common experience for first-timers to try a light task and then realize they've spent $3, instantly setting expectations for how easy it is to run up a large bill if you're not careful.

Also there's no way you can build a business without providing value in this space. Buyers are not that dumb.

Consider using Aider, and aggressively managing the context (via /add, /drop and /clear).

https://aider.chat/

I'd also recommend creating little `README`'s in your codebase that are mainly written with aider as the intended audience. In it, I'll explain architecture, what code makes (non-)sense to write in this directory, and so on. Has the side-effect of being helpful for humans, too.

Nowadays when I'm editing with aider, I'll include the project README (which contains a project overview + pointers to other README's), and whatever README is most relevant to the scope of my session. It's super productive.

I'm yet to find a model that beats the cost-effectiveness of Sonnet 3.7. I've tried the latest deepseek models, and while I love the price (nearly 50x cheaper?), it's just far too error-prone compared to Sonnet 3.7. It generates solid plans / architecture discussions, but, unlike Sonnet, the code it generates often confidently off-the-mark.

1 - https://github.com/plandex-ai/plandex

Also, a bit more on auto vs. manual context management in the docs: https://docs.plandex.ai/core-concepts/context-management

In a brownfield code base, I can often provide it reference files to pattern match against. So much easier to get great results when it can anchor itself in the rest of your code base.

This is a popular workflow I first read about here[1].

This has been the most useful use case for LLMs for me. Actually getting them to implement the spec correctly is the hard part, and you'll have to take the reigns and course correct often.

[1]: https://harper.blog/2025/02/16/my-llm-codegen-workflow-atm/

I wonder if the next phase would be the rise of (AI-driven?) "linters" that check that the implementation matches the architecture definition.

Everything old is new again!

For example it (Gemini 2.5) really struggles with newer ecosystem like Fastapi when wiring libraries like SQLAlchemy, Pytest, Python-playwright, etc., together.

I find more value in bootstrapping myself, and then using it to help with boiler plate once an effective safety harness is in place.

I've cancelled my copilot subscription last week and when it expires in two weeks I'll mostly likely shift to local models for autocomplete/simple stuff.

That said, months ago I did experience the kind of slow agent edit times you mentioned. I don't know where the bottleneck was, but it hasn't come back.

I'm on library WiFi right now, "vibe coding" (as much as I dislike that term) a new tool for my customers using Copilot, and it's snappy.

https://streamable.com/rqlr84

The claude and gemini models tend to be the slowest (yes, including flash). 4o is currently the fastest but still not great.

https://streamable.com/rqlr84

Stop fighting and sink!

But rest assured that with Github Copilot Coding Agent, your codebase will develop larger and larger volumes of new, exciting, underexplored technical debt that you can't be blamed for, and your colleagues will follow you into the murky depths soon.

You can tell because they advertise “Pro” and “Pro+” but then the FAQ reads,

> Does GitHub use Copilot Business or Enterprise data to train GitHub’s model? > No. GitHub does not use either Copilot Business or Enterprise data to train its models.

Aka, even paid individuals plans are getting brain raped

Oh cool!

> in well-tested codebases

Oh ok never mind

But also as a slightly deeper observation - agentic coding tools really do benefit significantly from good test coverage. Tests are a way to “box in” the agent and allow it to check its work regularly. While they aren’t necessary for these tools to work, they can enable coding agents to accomplish a lot more on your behalf.

(I work on Copilot coding agent)

They also have a tendency to suppress errors instead of fixing them, especially when the right thing to do is throw an error on some edge case.

On an unrelated note, it also suggested I use the "Strobe" protocol for encryption and sent me to https://strobe.cool which is ironic considering that page is all about making one hallucinate.

Oh wow, that was great - particularly if I then look at my own body parts (like my palm) that I know are not moving, it's particularly disturbing. That's a really well done effect, I've seen something similar but nothing quite like that.

I gave up trying to watch the stream after the third authentication timeout, but if I'd known it was this I'd maybe have tried a fourth time.

I’d love for this to blow past cursor. Will definitely tune in to see it.

I'm senior enough that I get to frequently see the gap between what my dev team thinks of our work and what actual customers think.

As a result, I no longer care at all what developers (including myself on my own projects) think about the quality of the thing they've built.

In coding agent, we encourage the agent to be very thorough in its work, and to take time to think deeply about the problem. It builds and tests code regularly to ensure it understands the impact of changes as it makes them, and stops and thinks regularly before taking action.

These choices would feel too “slow” in a synchronous IDE based experience, but feel natural in a “assign to a peer collaborator” UX. We lean into this to provide as rich of a problem solving agentic experience as possible.

(I’m working on Copilot coding agent)

I'm all for new tech getting introduced and made useful, but let's make it all opt in, shall we?

Good to see an official way of doing this.

1. https://github.com/asadm/chota

I kind of love the idea that all of this works in the familiar flow of raising an issue and having a magic coder swoop in and making a pull request.

At the same time, I have been spoiled by Cursor. I feel I would end up preferring that the magic coder is right there with me in the IDE where I can run things and make adjustments without having to do a followup request or comment on a line.

Wait, is this going to pollute the `gh` tool? Please tell me this isn't happening.

Sure! How can I help you?

Now Microsoft sits on a goldmine of source code and has the ability to offer AI integration even to private repositories. I can upload my code into a private repo and discuss it with an AI.

The only thing Google can counter with would be to build tools which developers install locally, but even then I guess that the integration would be limited.

And considering that Microsoft owns the "coding OS" VS Code, it makes Google look even worse. Let's see what they come up with tomorrow at Google I/O, but I doubt that it will be a serious competition for Microsoft. Maybe for OpenAI, if they're smart, but not for Microsoft.

https://developers.google.com/gemini-code-assist/docs/review...

Definitely not Google Code, but better than Cloud Source Repositories.

https://githubnext.com/projects/copilot-workspace

It seems Copilot could have really owned the vibe coding space. But that didn’t happen. I wonder why? Lots of ideas gummed up in organizational inefficiencies, etc?

AMAZING

But the upgraded Copilot was just in response to Cursor and Winsurf.

We'll see.

[0] https://news.ycombinator.com/item?id=43904611

Especially now that Copilot supports MCP I can plug in my own custom "Tools" (i.e. Function calling done by the AI Agent), and I have everything I need. Never even bothered trying Cursor or Windsurf, which i'm sure are great too, but _mainly_ since they're just forks of VSCode, as the IDE.

Copilot Workspace could take a task, implement it and create a PR - but it had a linear, highly structured flow, and wasn't deeply integrated into the GitHub tools that developers already use like issues and PRs.

With Copilot coding agent, we're taking all of the great work on Copilot Workspace, and all the learnings and feedback from that project, and integrating it more deeply into GitHub and really leveraging the capabilities of 2025's models, which allow the agent to be more fluid, asynchronous and autonomous.

(Source: I'm the product lead for Copilot coding agent.)

That seemed to drop off the Github changelog after February. I’m wondering if that team got reallocated to the copilot agent.

Edit: From the TFA: Using the agent consumes GitHub Actions minutes and Copilot premium requests, starting from entitlements included with your plan.

[0] https://docs.github.com/en/copilot/managing-copilot/monitori...

Copilot literally can't push directly to the default branch - we don't give it the ability to do that - precisely because we believe that all AI-generated code (just like human generated code) should be carefully reviewed before it goes to production.

(Source: I'm the product lead for Copilot coding agent.)

To me, this reads like it'll be a good junior and open up a PR with its changes, letting you (the issue author) review and merge. Of course, you can just hit "merge" without looking at the changes, but then it's kinda on you when unreviewed stuff ends up in main.