Here’s part two in our series exploring the use of LLMs to generate successful FOIA requests.
This time, I am joined by Adam Marshall, director of national litigation at the Reporters Committee for Freedom of the Press.
Adam takes a different position from our previous guest, Lauren Harper, Daniel Ellsberg Chair on Government Secrecy with Freedom of the Press Foundation. You can listen to Part 1 here.
This is important information for those of you who want to stay on top of what ICE and DHS is doing in our community given their sharp uptick in activity that started in July.
A real hero—pictured above—is documenting the increasing presence of ICE in Monroe and in Kiryas Joel.
Full Transcript
[Editorial note: I did attempt to use Claude 4.5 Haiku along with Duck.AI to create this week’s transcript. I am desperate to find ways to save time with anything that’s not investigating and writing, and I absolutely hate doing these transcripts. So, let’s see how this goes. Just note that this transcript may be slightly different from the audio version above.]
BJ Mendelson: Adam, thank you so much for joining us today. Would you like to take a moment to introduce yourself to our audience?
Adam Marshall: Sure, my name is Adam Marshall. I am the director of national litigation at the Reporters Committee for Freedom of the Press.
BJ: And can you tell us just a bit more about what the Reporters Committee does?
Adam: Yeah, so the Reporters Committee is a nonprofit. We are based partly in DC, but we also have attorneys in states across the U.S. We are primarily a bunch of attorneys that provide pro bono legal services to journalists and newsrooms in the United States. We do a lot of public transparency work—records requests, court access, that type of thing. But we touch on all kinds of issues at the intersection of journalism and law, to try to support reporters and newsrooms as they do their job.
BJ: And I think that’s a perfect fit for what we want to talk about today. Our goal at the Monroe Gazette is to train people to report on news within a ten-mile radius of their house. That’s our overarching desire. And sometimes it bumps into this question of large language models and whether they’re effective or useful. So before we get into it, I’m just curious: up until this point, what has been your interaction with large language models when it comes to the work that you do?
Adam: I mean, I believe when I first typed anything into ChatGPT—years ago when it first came out—the first thing I ever did was ask it to draft a FOIA request, actually.
BJ: Nice.
Adam: Because I was like, this could be interesting. I wonder what it will do. So from the very beginning of public access to these chat interfaces for LLMs, I’ve been testing them out and thinking about them—where they could be useful, where they do helpful stuff, where they are not as helpful or maybe even affirmatively unhelpful. So yeah, it’s been several years now that I’ve been tinkering with it and theorizing about the intersection between FOIA and these tools.
BJ: Generally speaking, do you think they’re useful for generating a FOIA request?
Adam: I’m going to say the lawyerly thing, which is: it depends.
BJ: Sure. Yeah.
Adam: It completely depends on what model you are using, what instructions you give it, and what you are asking about. I think that in general, if you just ask an LLM—certainly some of the earlier models, but continuing through more recent models—to generate a FOIA request for records about X or Y or Z, it will generally produce something that looks like a FOIA request if you don’t dig into the details too much.
But the descriptions of the records in the request are often quite poor. It often misses a lot of key things that requesters should address in the request. And so I’ve been pretty skeptical about just taking the output from LLMs and using it as a FOIA request.
Where I do think there is potential is to create custom RAGs—retrieval-augmented generative tools—that look at a corpus of government record descriptors to try to identify potential records to ask for in a records request.
I think that’s where the real magic of LLMs can be helpful. Because one of the challenges a requester faces in describing records is that agencies are just often like black boxes. There’s so much that goes on, but it’s so hard for a layperson or even a journalist who’s interested in something to figure out exactly what to ask for. And where I think an LLM can be helpful is translating the kind of underlying semantic content of what you’re interested in into the bureaucratic language that agencies might use.
So if I’m interested in, say, a contract at an agency, I might not know how the agency goes about describing those types of contracts or related documents. But if I can use an LLM to examine the agency’s forms and regulations and all kinds of stuff, the LLM might be able to say, this is what you’re interested in. This corresponds to Form 274/D. And that’s really what you should be focused on. And then if I go out and verify that Form 274/D is in fact what I’m interested in, then I think that is a huge benefit to the requester because they can be really precise and the agency then knows exactly what they’re talking about.
BJ: Yeah, and I think you kind of hit some of the problems head-on. And so I’m wondering: what would you advise people if they’re given a choice? Would you say that life before LLMs—is it still maybe a matter of having better success using something like FOIA Wiki, for example, which I know you guys are absolutely familiar with and we can definitely talk about?
Adam: Yeah.
BJ: Would you steer people toward more of those resources than using the LLM? Let’s say you have a new journalist coming to you. Would you steer them toward more of those resources?
Adam: I don’t think there’s a huge benefit to using an LLM for the other parts of the request because of a few reasons. One is that the other trappings of the request seem to be pretty standard, right? It’s like, who are you? How much are you willing to pay for this? What is your address? That type of thing.
And then there are other parts of the request that require some kind of argument. For example, the requester might argue that they’re entitled to expedited processing or a fee waiver. And in those contexts, I don’t think that the LLMs often do a great job in making a convincing argument. I don’t know. The models are changing all the time and getting better and more sophisticated. So this isn’t to say they’re never going to be helpful.
But I think the real value add is in using them to identify very particular types of records that would otherwise be difficult for a requester to know about.
BJ: That takes us perfectly into the example we posted where I found that if not for Project Saltbox and if not for feeding that specific Project Saltbox link into the LLM, I don’t think it would have been able to help me identify the records concerning 800 Corporate Boulevard. But I did find that Claude in particular was helpful in, as you pointed out, saying, okay, well this is specifically what you need and what you don’t need.
Adam: Yeah. And so I was just looking at the preface to your request. It looks like you used Claude 4.5 Haiku via Duck AI.
BJ: Yes. That’s right.
Adam: And you know, I’ve also tended to find—to be honest, I have not played around a lot with Haiku. I think in general the bigger parameter models tend to give much better results. So with respect to Claude, I think I’ve only looked at it using Sonnet and Opus. I don’t think I’ve experimented with Haiku.
BJ: I don’t want anyone to take this as an endorsement for Anthropic or Claude. Yeah.
Adam: No. No.
BJ: I certainly don’t. And I’m one of the plaintiffs in the Bartz v. Anthropic case, so I can factually say they stole my stuff to train their model. So our interest in promoting these things is just: how can we arm everyday citizens to act like reporters, and how quickly can we onboard them? And is this better than some of the other things? Can you tell us a little bit about FOIA Wiki and maybe why that would be the more superior thing for people to go to?
Adam: Yeah, well, I mean, I certainly like it because it’s a resource that the Reporters Committee hosts and that other groups have contributed to over the years. I’m not saying it’s the best one out there, but it is certainly something that we like to send journalists to when they have questions about FOIA. I think that federal FOIA in particular is not always easy to use. And that’s a shame because these laws exist for the benefit of the public at large. But for better or for worse, we have sixty years now of case law and regulations and implementation of FOIA. And I do think it behooves people to understand how the law works, both as a legal entity and as a practical matter.
And to do that, you just have to get your hands dirty a little bit. You have to read agency regulations, understand what are the trade-offs between the scope of your request and how quickly you need it, understand how to make an argument about timeliness or public interest.
I do think that flexing your brain muscles around these things helps ultimately produce better requests. And there are a lot of tools that can help us produce better work product, but at the end of the day, I don’t think there are a lot of shortcuts that can be taken. So I think reading about FOIA from a whole variety of sources, including from the government frankly—the DOJ guide to FOIA—to better familiarize yourself with how the law works is always a good idea.
BJ: Yeah, and I think a good example of using your brain is—I’m sure you’ve heard by now about ICE’s Office of Professional Responsibility.
Adam: Yes.
BJ: Yeah, which Wired recently reported on, an incident in Syracuse, among others, where a poll watcher was approached by two ICE agents who wanted her to sign a form basically saying that she’s been warned essentially not to post nasty things about ICE on the internet. So all of us this week are learning about this office and what it’s supposed to do versus what it’s currently doing. And so using our brain, like what would you suggest someone who wants to be a reporter? How would they even begin to tackle something like that using FOIA?
Let’s say we’re going to take the LLMs out. Let’s say the LLM is not conducive, and we’re going to use some of these other resources out there. How would we approach that?
Adam: So I think, any time anyone is drafting a FOIA about a particular subject, I would just try to read everything that I can about it. Find all of the reporting you can. But then also start to dig into an agency and an office’s organizational chart. That can often be interesting. Start digging into the agency regulations to figure out how different processes flow throughout the agency. What types of records might exist or be required to be created at different points in those processes that you might ask for.
You know, looking at examples of prior requests—MuckRock, for example, is a great resource here. You can see what other people have previously requested from that agency or that office, and then see if those might be updated or built upon in terms of what you’re interested in now. Looking at press releases or statements or anything that the agency itself is affirmatively putting out could provide a basis for asking for records. You could also ask for communications between specific people that mention a word or phrase between certain dates. There are lots of different ways to go about it.
But I think a trap that people often fall into when making a FOIA request is they’re like, okay, I’m interested in this topic, right? Topic X or Topic Y. And then they submit a FOIA request that says, I want any and all records related to Topic X. And that’s not going to work under federal FOIA because it’s not sufficiently specific. You have to drill down more into what types of records you’re actually looking for. And so that’s where it’s really helpful to understand how the agency works, understand what people are involved, what offices are involved, what processes are involved, so you can be more specific in the request.
The Monroe Gazette is ad-free and paywall-free, covering local news in Monroe and surrounding towns as well as county, state, and national issues that impact those residents.
Want to help keep the lights on? We’re looking to recruit 995 paid annual subscribers between now and June 24, 2027. If you’re new here or already a free subscriber, press that beautiful button below to upgrade.
BJ: Let me ask you about what happens if you get something that’s completely redacted. So for people listening, we put in a FOIA request for the EPA where we wanted to know what instructions they received upon Lee Zeldin becoming the head of the EPA in terms of budget cuts, DOGE, and staff reductions. And we received a bunch of stuff that was redacted.
But unlike with FOIL, I’m not at all familiar with how you appeal a FOIA request. So is this something we would go to an LLM for, or what resources are out there for us to look into how to handle either redactions or denials of requests?
Adam: Yeah, so under federal FOIA there’s what’s called an administrative appeals process, which is basically you appeal higher up within the agency and have them take a second look at it. The agency will tell you in the denial letter who you send that to. Requesters will usually have a minimum of ninety calendar days to submit an administrative appeal. And so there are exemptions in federal FOIA scattered throughout the United States Code that allow agencies to withhold either specific records or types of information. And if the agency asserts one of those exemptions, you basically have to come up with a legal argument for why those don’t apply.
There are lots of guides out there that provide requesters information about those exemptions and potential arguments that you might make. So the FOIA Wiki is one of them, but there are lots of guides out there. And the government even puts out its own guide on FOIA exemptions. So in crafting legal arguments, I think it depends upon what specific tool you’re using. But I think there are a lot of potential pitfalls and dangers with respect to using LLMs because they do tend to hallucinate legal citations in a lot of cases, misunderstand cases, or just not really get the issue.
BJ: Right.
Adam: And so I would be extremely cautious in using any of these tools to craft administrative appeals.
BJ: Yeah, and we can flag for readers as well. We’ve talked a lot about FOIL, the New York State equivalent, on The Monroe Gazette. And yeah, we’ve definitely seen cases that have been made up by Claude and by some of the other ones.
Adam: Yeah.
BJ: So what would you say? Like, a new reporter has come to you and they’re like, give me everything I need to know about FOIL and FOIA. What are the tips and tricks that you always start with?
Adam: I mean, a lot of it is about how to craft descriptions of the records you’re after. I spend a lot of time with reporters on that, and a lot of it covers things we’ve already talked about. But there are other procedural things you need to know when it comes to FOIA in particular. Like if you are a journalist, it certainly behooves you to know how to make a request for a fee benefit as a member of the news media. You need to know how to make a fee waiver request if you’re interested. You need to know how to ask for expedited processing. There’s just a lot of minutiae that go into drafting a really good FOIA request, and there are a lot of branches that you have to choose between in drafting a request that affect what it’s going to look like. And so when I train journalists on FOIA, a lot of it is just walking through the basics: who you can FOIA, how do you craft a description of what you’re asking for, and then walking through all of these different branches to figure out or to at least get the lay of the land in terms of all the things you should be thinking about as you’re drafting the request. And you know, a lot of times state or local government agency public records requests can be a lot simpler to draft.
But for federal FOIA, again, for better or for worse, it ends up being a pretty technical exercise.
BJ: Right.
Adam: Yeah.
BJ: And that’s what led us to the LLMs in the first place.
Let me ask you real quick about—again, going back to we want people to be journalists covering news within ten miles of their house. So how would they apply for a fee waiver, if at all?
Adam: You can go look at the FOIA Wiki. We have a whole section on fee waivers. There are two general prongs that you have to make an argument for with respect to a fee waiver. You have to show that there is a public interest in the information that is likely to contribute significantly to the public understanding of the operations or activities of the government. And then the second one is you have to show that it’s not primarily in the commercial interest of the requester. And if you are gathering information to distribute it, especially to the public for free, the latter category is easier to make an argument for. And then on the first prong, you need to be showing: what is it about the federal government that these records are going to illuminate, and how is it additive to what is already kind of known about what the federal government is doing?
And for both of these, yes, there are cases that you could cite. But I think a good common-sense argument is going to go a lot farther than a really chock-full legal brief.
BJ: Yeah, simple seems to be better, right? Like that seems to be at least with this. And is that true as well for expedited processing? Like have you ever seen that actually granted?
Adam: Yeah, so requests for expedited processing are granted. The rate varies agency to agency. Not many requests for expedited processing are granted. But they certainly are sometimes. Expedited processing can be a little more involved because you might need to read the statute in conjunction with the agency regulations.
So there are two specific ways in FOIA—the statute itself—that you can qualify for expedited processing. And then the agency regulations can basically add on additional circumstances in which you can get expedited processing. And then, same thing for these: you can look up case law, certainly on the two main situations in which you might get expedited processing. But a lot of it is just really solid logical arguments.
