Meet the experts who power Synack’s strategic security testing platform. Our Synack Red Team unites over 1,500 of the world’s most skilled and trusted security researchers, who work with patented technology to deliver best-in-class offensive security testing on a continuous basis.
Responsible Disclosure and Bug Bounty Programs: Webinar
Guest:
Adam Logue
Published:
Updated:
Duration:
25 min
Episode summary
Adam Logue, Independent Security Researcher and Synack Red Teamer, discusses his experiences with responsible disclosure and bug bounty programs, and provides a fascinating technical deep dive into a vulnerability he found in Microsoft 365 Copilot during a client-facing engagement.
Listen to the episode
S4 E6 · Responsible Disclosure and Bug Bounty Programs: WebinarWE'RE IN! Podcast · 25 min
Key takeaways
Agentic AI widens data exposure Because M365 Copilot can fetch recent emails and Teams chats, an indirect prompt injection in a shared document can pull that data into an attacker-controlled link.
Trusted interfaces make phishing believable A simple Mermaid flowchart box with a lock emoji looked convincing as a login button because it appeared inside the Copilot window, and a fake sign-in image hid the exfiltration.
Report platform flaws to the vendor The client could not fix the issue itself, so Logue reported it through Microsoft's MSRC program and published his blog post only after the fix was confirmed.
Learn AI testing by experimenting Logue recommends playing with LLMs that have many tool integrations, studying Johann Rehberger's blog, and practicing on Damn Vulnerable LLM Agent, since old tricks like ignore previous instructions rarely work now.
Smaller crowds mean fewer duplicates On SRT targets he may compete with 14 to 16 researchers instead of a thousand, and visibility into submitted endpoints saves time on duplicate reports.
Notable quotes
…there's more than just false information that can come from using AI in your day to day work.
Again, it's not super believable, but because of the whole context of it being right in the middle of the whole M365 Copilot window, it might trick some people as well.
…the best way to learn is just by playing with an AI or playing with different LLMs and finding the ones that have all these extra integrations…
Questions this episode answers
How did the Microsoft 365 Copilot exfiltration vulnerability work?
A document with an indirect prompt injection made Copilot fetch the user's recent emails and render a Mermaid diagram styled as a login button. Clicking it sent the emails, hex encoded in the URL, to the attacker's server while showing a fake sign-in image. Read this section
How was the Copilot vulnerability found and reported?
Logue found it while testing a client's M365 Copilot rollout in a hybrid Active Directory environment. Since the client could not fix it, he reported it through Microsoft's MSRC program and published after the fix. Read this section
How can I get started with AI and LLM penetration testing?
Logue suggests experimenting with LLMs that have many tools and integrations, asking them what tools they can access, and using gradual techniques like crescendo rather than direct override prompts. He also points to Johann Rehberger's blog and Damn Vulnerable LLM Agent. Read this section
How can a new pentester join the Synack Red Team?
There is a waiting list, but the SRT Pathways program is one of the better routes. Hack The Box challenges and certifications can also fast-track access to specific techniques on the platform. Read this section
What should employees watch for when using Copilot?
Logue says employees should be trained to know that AI tools can be hijacked by prompt injection, not just give false information. Mason adds that a login prompt while you are already signed in is a warning sign. Read this section
About the guest
Adam Logue is an independent security researcher who, at the time of recording, works as a full-time freelance penetration tester and has been a member of the Synack Red Team since 2017. He started his career as a web application penetration tester for a hospital group and practiced responsible disclosure before joining bug bounty platforms. He found a data exfiltration vulnerability in Microsoft 365 Copilot during a client engagement and published his findings after Microsoft fixed it.
Hello, and welcome to WE'RE IN, the Synack podcast. I'm Josh Mason, a solutions architect here at Synack. And with me, I've got Adam Logue, one of our Synack Red Teamers. Today, we're going to... I'm gonna be interviewing Adam. Adam has recently found a Copilot zero day and released the the notes on it on his personal blog after working with Microsoft on, patching the vulnerability. But before we get into those specifics, Adam, tell us a little bit about yourself.
Yeah. So my name is Adam Logue. Again, I've been with Synack on the SRT platform for, since 2017. So almost a decade now coming up on it. So seven, eight years. I've been, you know, before that, trying to do bug bounty in the most responsible way possible. But back then they didn't really have a whole lot of bug bounty programs. Was a lot, you know, just responsible disclosure. And even then sometimes you'd be faced with, you know, some, some companies that didn't actually take too lightly to, you know, vulnerabilities found in their products. So, I'm definitely happy that over time, bug bounty and vulnerability disclosure programs have definitely progressed and made it a little more safe for people to be able to disclose these vulnerabilities.
So as far as bug bounty goes, I kind of naturally fell into that. So if we back up a little further, I got my first, I guess, boy job, you could say out of college as a web application penetration tester for a hospital group. And during that time I learned a whole lot of web application pentesting fundamentals and I figured, oh, I can apply this in the real world and look for vulnerabilities under applications. So, one of my other friends using SRT as well, know, we ended up like downloading a bunch of different phone apps and we started like connecting them through Burp Suite and figuring out, oh, Hey, like we can see stuff we're not supposed to here. And then we're like, well, now what? We should report it to them. So we would end up trying to reach out to them, either emailing them securitythecompanyin dot com or reaching out via LinkedIn, which is what we had to do for a couple of companies.
And, we ended up getting pretty lucky. You know, there are some people out there that would get you know, sent a whole bunch of cease and desist letters and like there'd be legal trouble and stuff. And we weren't trying to ruffle any feathers or anything. We just wanted to disclose these things responsibly. For the most part we had pretty great open channels with these companies and after they were sorry, after they were fixed, we were able to publish those results to our blog posts. So that was sort of the precursor to Bug Bounty. Then after that I ended up joining HackerOne and then shortly after that Synack. So definitely enjoying the Bug Bounty life.
So currently, I... Yeah. Currently, I'm a 100% freelance and I do, you know, ten ninety nine penetration testing for a whole bunch of different companies. Oftentimes, you know, during the busier times of the year, Q4 here, we get a whole bunch of overflow and they need penetration tests done. So I'll do those. And that sort of helps guarantee me some like, forty hour a week work, you know, flat rate. And then, you know, I continue doing the bug bounty stuff and hunting on the side as well as extra little income there.
Sure. I imagine a lot of folks are realizing they forgot their PCI needs to be done before the end of the year. At least that's what it was when I was at pentesting firms in the past.
Yeah. A lot of times, these companies have budgets and it's use it or lose it and they realize, oh no, I have to use it. So they, you know, just try to get their checkbox and usually gets sent to me for that kind of stuff.
Excellent. You found an exploit or you found a vulnerability in Copilot. Was it during a a pentest or during an SRT, mission that you found that? Or was it just on your own research?
Yeah. So that was actually during a client facing engagement. The client asked me like, hey, we're looking to integrate, you know, M three sixty five Copilot into our, know, active directory hybrid cloud environment, and we wanna make sure that it's actually secure and make sure that the security controls that they say are in place are actually in place. So they pretty much made it sort of open ended, said, alright, go ahead and test this. Here's the CoPilot starting point. Go. And so I said, alright, let's see what we can do here. So a lot of the testing was, you know, make sure that the CoPilot portion that can access the m three sixty five tenant data couldn't access, you know, like directly other stuff on the Internet and then make sure that the Internet facing portion couldn't access any of the internal tenant stuff.
And during that time, I figured out, oh, you know, like, we can actually grab this data because with, you know, the whole concept of agentic AI being able to access all these APIs and tools and being able to fetch my recent emails or fetch my recent Teams chats, like it has access to all this corporate data. And I was like, man, wouldn't it be cool if we found a way to actually exfiltrate this kind of stuff? So I ended up doing a little bit of research to see what was out there for that kind of thing. And there wasn't a whole lot, you know, at the time, I guess. There was one specific person in particular. He goes by the name wunderwuzzi. And let's see here. I think his actual name is Johann Rehberger, but he came out with this embracethered.com blog of his.
And then he had this whole month of AI bugs where he actually, for that whole month of, I think it was August of this year, he went and tried to find as many AI bugs as he could and just sort of powered through that. And a lot of his stuff was really cool, but didn't necessarily correlate directly to the M365 Copilot environment. But I did find an interesting similar concept that he had with mermaid diagrams. Now I was actually able to find the mermaid diagram trigger for this where I can actually exfiltrate data by creating this mermaid diagram. Essentially all it is is just a little flowchart box. And I found that I could make it look like a login button.
So in his research, he was able to actually like overlay an image and make the whole login box look all nice and pretty. I didn't necessarily have that luxury because of the restricted environment. So I just added a lock emoji and made it blue and said, this looks good enough. And, I was able to then have, you know, the URL when you click it, it would make a request to the server. So I started looking to chain this together with, you know, fetching the recent emails, including that data hex encoded in the URL. And, you know, as soon as that link is clicked, all that data goes outbound. But then I realized, okay, this data exfiltration technique, it doesn't necessarily result in like a Turing complete type of exploit. So I needed to sort of combine it with something like prompt injection to be able to actually, you know, make it something that was close to like a one click.
I was trying to go for a zero click, but unfortunately I couldn't figure that out. And I wanted to make sure I could publish this in a timely manner because again, this was a client facing engagement. I had a limited testing timeframe with their environment. And then I also was using my own personal Azure environment for also playing with this as well. So once I found the penetration test, reported it, and then I had to go through and explain to them, hey, we need Microsoft to fix this because there is no actual fix in place here. So then I figured, okay, I have to report it to Microsoft. So I just closed it through the MSRC program that they have and they were able to fix it. And once they confirmed the fix, then I was able to publish the blog post.
Yeah. It it is sort of hard because a lot of people, they blindly trust that, oh, this is coming from a m three sixty five application. Therefore, it must be a legitimate, you know, login button. And again, with the whole CoPilot concept, it's, oh, I can have this summarize a document for me. You know, there's been a whole push for that. You start to see it in the current M365 suite and things like Outlook and Microsoft Office, Word and Excel and PowerPoint, it'll say, you know, like, hey, summarize this for me and I can do this other cool stuff for you too. And they're trying to push the integration. And I think with that, there's not a whole lot of people realizing the ability to have prompt injection take over that and start to perform arbitrary tasks and queries that they didn't necessarily intend.
So I think that from a security perspective, it's definitely important to train, you know, employees to be mindful and aware that, there's more than just false information that can come from using AI in your day to day work.
Yeah. I think of... Sometimes we install video games or software on our computers and we'll get a UAP, a pop up asking, are you sure you wanna do this? Or asking for admin privileges to do a thing, and some of us are, we've been doing it our whole lives, just saying yes because we planned on needing to do that to do the thing that we asked it to do. But if it's just having it, you know, Copilot read an Excel file and, you know, give us the information from it, that's something that shouldn't be prompting us for a login since we're already in. So having that sort of awareness, you know, if I open an email and I'm already in Outlook, I shouldn't be having to prompt it for a login or for a user access control.
Yeah. And again Yeah. I tried to make it look believable. So during my exploit, I realized that when you actually clicked on the link, it would show the response of the web page that was loaded. So if I use like a a Burp Suite Collaborator payload, then it would just show like the actual little, like, string in the response. And then after a few seconds, would go away. So I thought, wouldn't it be neat to actually try to social engineer people a little bit more and make them feel like it was a little more legitimate by returning a image of the redirecting you to the login, page image. So I just kinda like found that, took a screenshot, edited my private Burp Collaborator server to then have anything going to slash login, you know, return that image, and then that was kinda how I pieced that together.
Yes. So I will zoom in here. So when you ask Copilot, you know, summarize this document and you drag and drop in a Excel spreadsheet, It also like hyperlink to it. During this process, it also uploads it to OneDrive as well, which I thought was really interesting. So we hit go and specifically we see it says, okay, I'll search for recent emails, but this actually pops up and it goes away pretty quickly. So a lot of times people will miss that if they're not paying attention. And it says to view the content of the document titled financial report, please log in using the secure link below. Now this is the part where it's actually executing the arbitrary indirect prompt injection and I'm fetching all of the recent emails for that user and, you know, it's building this mermaid diagram right here.
So, again, here's this, you know, single box flowchart that we just, you know, made into a login button with our pretty styling. And because we can add any image overlays or anything, we just use this lock emoji right here. Again, it's not super believable, but because of the whole context of it being right in the middle of the whole M365 Copilot window, it might trick some people as well. A of us are
Yeah. So when the user clicks that it loads the HTTP response here, which again, this is just a image of a trying to sign you in, know, thing that I found during a Microsoft M365 sign in, took a quick screenshot, and then just made that the actual image that would return in the response when you made a request to that specific endpoint. Now specifically in the URL, it would include a c equals and then have all the hex encoded data from the emails included in that web request to the attacker server here. And then I noticed that this response box would go away after a few seconds and then people would think, oh, no, nothing. They would think, oh, there's not really anything going on different here.
So now if we go over to our Burp Suite and we can see our incoming HTTP request here, we can actually take a look at the URL. And so the URL that the login button made a request to was this Burp Collaborator URL for my private collaborator server. And then it had the slash login, which would force collaborator to return that Windows fake, you know, logging you in image. And it would include this c parameter and include all the hex encoded data from the mermaid diagram from recently fetched emails there. So when we highlight this and we take it and we put it into burp decoder and then we decode it as hex, you can see it's all clear text stuff. And again, this is just like bullet points, but you can see the concept of being able to return things like MFA tokens or, you know, password reset links and things like that, where if it's a targeted attack, you're able to actually extract this data from, you know, corporate users inbox and then have it sent to the attacker server.
That's pretty awesome. Yeah. If you were to send that to a CFO or an accountant, you could get some account information or invoice information. Think of that scammer who got millions from Facebook and Google by just invoicing them for years and nobody checked. Sure. And just paid the invoices. I mean, was eventually arrested, but for a while.
Yeah. It's threats like that, that nobody thinks about. We're always worried about protecting against breaches. But there's a lot of attacks that don't necessarily require someone hacking all the way into everything. So you were able to put this together without years and years of AI pentesting experience. You found that month of bugs and a few other pieces and then your experiences pentesting for over a decade. If someone else wanted to start getting into, you know, pentesting on AI, what would you recommend?
Yeah. So, the AI space, again, it's very new, but there are a lot of different bugs being found and reported, like, you know, even more recently as we see more and more features and functionality being integrated, we're finding that common things like human in the loop behavior that we expect to have in our AI prompts can be circumvented sometimes under certain conditions. And if you're able to leverage those, you're able to, a lot of times, make it do things that shouldn't do. So one of the common trends I found recently is if there are specific LLMs that are working with files for things like coding, A lot of developers are using Vibe coding in their day to day work for having the LLM write the code for them.
They're able to run test cases and check that, but there's specific security constraints. And if you're able to get the AI to override its own security constraints, you're able to a lot of times circumvent and bypass a lot of these effective measures that were in place. So we found that with a lot of things like the VS Code implementation with a lot of these MCP or model context protocol integrations with these different AIs.
Yeah. I'd highly recommend, you know, honestly, the best way to, you know, learn is just by playing with an AI or playing with different LLMs and finding the ones that have all these extra integrations, all the different tools that they can work with, you know, asking like what tools you have access to. A lot of times that helps open up the attack surface and sometimes it requires finessing it a little bit. You know, a lot of the older tricks that used to work, you know, such as saying like ignore all previous instructions or disregard all previous instructions, do this instead, that doesn't work as well anymore. You kind of have to use the crescendo technique and kind of pivot the conversation in the way you want it to go. And again, you don't always have that capability in stateless conversations where it can only get like one shot to give it all the information and it'll give you just a response and it doesn't remember the previous stuff, but you know, what capabilities it has, sometimes you can find ways to circumvent the standard operating procedures and circumvent its instructions to avoid disclosing sensitive data and that kind of stuff.
If someone's new to pentesting and they've... They found this because they heard about, you know, a Copilot vulnerability and they're tuning in, Do you have any recommendations for where, you know, if Adam Logue was getting started today, where you would recommend that he start, you know, his path off?
I mean, I would go and, you know, one look at Johann's blog. You know, he has a whole bunch of different techniques, a lot of them are pretty bleeding edge as far as what he did against a lot of these. And, you know, they're all patched as far as those go, but it can definitely give a lot of ideas towards other types of attacks. Other things to look at are some of these LMs like you got like DVLA, damn vulnerable LLM agent. It's sort of like a LLM agent vulnerable by design. It has different sort of like a little CTF type of thing where you have to get the different flags and it's a great way to be able to play with it and kind of learn what works and what doesn't work for different prompt injections.
That's pretty cool. I hadn't heard of that one. What about for like the brand new person who, you know, they wanna be a pentester, they wanna be on SRT one day, and they're just getting started. It's a different world than when I got into cyber. It's a different world than when you got into cyber. What would be your recommendation on that front?
So there's obviously quite a big waiting list. But if you work on one of the SRT Pathways programs, that's one of the better ways to do it. There's different hack the box challenges or different certifications that you can get. And if you can prove like, Hey, I have the certification. You can get fast tracked for, those specific techniques on the platform.
So it's, it's definitely not as easy as, you know, signing up for a HackerOne account because you do have to go through the vetting process. But one of the nice things about the, Synack Red Team is you have fewer people looking at an overall target and so there's less competition. And even then, you know, on a specific target, you might only have like maybe 14 or 16 people instead of a thousand people that you're working against. And as far as platforms go, having the analytics to see what other endpoints have been submitted definitely reduces time for dupes and everything like that, where it'd be wasted time to write up a report only to get rejected because someone else found it first.
Those are exactly what I hear from a lot of other SRT members as well. Thanks, Adam. This has been a lot of fun chatting with you. Thank you for sharing your research and a little bit about your path. I hope others have found this interesting as well. And if you have, like, share, Please point others this direction, and we'll see you on the next one.
