How I automate my own job at Hugging Face using agents — Niels Rogge, Hugging Face

>> Okay. >> Okay. All right. All right. All right. Hello everyone. Hello everyone. Hello everyone. Thanks for coming by. Thanks for coming by. Thanks for coming by. Today I'll talk about how I automate my Today I'll talk about how I automate my Today I'll talk about how I automate my own job at Hugging Face using agents. own job at Hugging Face using agents. own job at Hugging Face using agents. Um short introduction. I'm just uh Niels Um short introduction. I'm just uh Niels Um short introduction. I'm just uh Niels from Belgium, the land of beer, fries, from Belgium, the land of beer, fries, from Belgium, the land of beer, fries, and chocolate. I studied at KU Leuven, and chocolate. I studied at KU Leuven, and chocolate. I studied at KU Leuven, and I'm a machine learning engineer at and I'm a machine learning engineer at and I'm a machine learning engineer at Hugging Face for 5 years now. Uh today I'll talk about the community Uh today I'll talk about the community science team at Hugging Face, which is science team at Hugging Face, which is science team at Hugging Face, which is the team I'm part of. the team I'm part of. the team I'm part of. Uh then I'll talk about how I automate Uh then I'll talk about how I automate Uh then I'll talk about how I automate large parts of the community science large parts of the community science large parts of the community science team. And finally, I'll also discuss team. And finally, I'll also discuss team. And finally, I'll also discuss some other efforts uh that we do at some other efforts uh that we do at some other efforts uh that we do at Hugging Face. Hugging Face. Hugging Face. So, let's start with the community So, let's start with the community So, let's start with the community science team at Hugging Face. science team at Hugging Face. science team at Hugging Face. So, basically, this started when I was So, basically, this started when I was So, basically, this started when I was sent I was seeing like trending research sent I was seeing like trending research sent I was seeing like trending research uh passing by on GitHub. And a lot of uh passing by on GitHub. And a lot of uh passing by on GitHub. And a lot of times when I saw new interesting work, times when I saw new interesting work, times when I saw new interesting work, um the weights were not available on um the weights were not available on um the weights were not available on Hugging Face, sadly. Like, researchers Hugging Face, sadly. Like, researchers Hugging Face, sadly. Like, researchers use either Google Drive or they use use either Google Drive or they use use either Google Drive or they use GitHub releases.

GitHub releases. GitHub releases. They use Dropbox, they use Zenodo, or They use Dropbox, they use Zenodo, or They use Dropbox, they use Zenodo, or other servers to put their um artifacts other servers to put their um artifacts other servers to put their um artifacts on. And this hurts uh discoverability of on. And this hurts uh discoverability of on. And this hurts uh discoverability of their work. It's like not easily their work. It's like not easily their work. It's like not easily uh visible or discoverable. uh visible or discoverable. uh visible or discoverable. And when I then open a GitHub issue to And when I then open a GitHub issue to And when I then open a GitHub issue to say like, "Actually, you could put your say like, "Actually, you could put your say like, "Actually, you could put your weights on Hugging Face for free." Most weights on Hugging Face for free." Most weights on Hugging Face for free." Most of the time, people replied to me like, of the time, people replied to me like, of the time, people replied to me like, "Yeah, migrating the weights uh from "Yeah, migrating the weights uh from "Yeah, migrating the weights uh from Google Drive to Hugging Face actually Google Drive to Hugging Face actually Google Drive to Hugging Face actually makes perfect sense." makes perfect sense." makes perfect sense." So, yeah, the community science team can So, yeah, the community science team can So, yeah, the community science team can also uh be described as the Google Drive also uh be described as the Google Drive also uh be described as the Google Drive to the hub team. to the hub team. to the hub team. Um why? Because on Hugging Face, we have Um why? Because on Hugging Face, we have Um why? Because on Hugging Face, we have these paper pages, these paper pages, these paper pages, uh and every single paper is from uh and every single paper is from uh and every single paper is from archive. And then on the right side, you archive. And then on the right side, you archive. And then on the right side, you can basically list the linked artifacts, can basically list the linked artifacts, can basically list the linked artifacts, like the the linked models or data sets. like the the linked models or data sets. like the the linked models or data sets. So, people can easily reproduce your So, people can easily reproduce your So, people can easily reproduce your paper or find the models or data sets. paper or find the models or data sets. paper or find the models or data sets. So, yeah, you can see them on the right So, yeah, you can see them on the right So, yeah, you can see them on the right side. side. side. Um this improves the discoverability of Um this improves the discoverability of Um this improves the discoverability of your work because we have these metadata your work because we have these metadata your work because we have these metadata tags or filters on the app, so you can tags or filters on the app, so you can tags or filters on the app, so you can easily find, for example, depth easily find, for example, depth easily find, for example, depth estimation model, estimation model, estimation model, an LLM if you're interested. You can an LLM if you're interested. You can an LLM if you're interested. You can find them by language. You can tag them find them by language. You can tag them find them by language. You can tag them with the library they are compatible with the library they are compatible with the library they are compatible with and so on. So, this improves the with and so on. So, this improves the with and so on. So, this improves the discoverability of your work. So, these discoverability of your work. So, these discoverability of your work. So, these are, yeah, the metadata tags that you are, yeah, the metadata tags that you are, yeah, the metadata tags that you can add to every single model on Hugging can add to every single model on Hugging can add to every single model on Hugging Face or every single data set.

Face or every single data set. Face or every single data set. So, yeah, this is like the main problem So, yeah, this is like the main problem So, yeah, this is like the main problem that we saw, like, lots of people, lots that we saw, like, lots of people, lots that we saw, like, lots of people, lots of researchers are like using of researchers are like using of researchers are like using third-party services to publish their third-party services to publish their third-party services to publish their work. We have the Hugging Face platform, work. We have the Hugging Face platform, work. We have the Hugging Face platform, which is like a centralized place where which is like a centralized place where which is like a centralized place where people can find machine learning people can find machine learning people can find machine learning artifacts. artifacts. artifacts. It also improves with documentation It also improves with documentation It also improves with documentation because you can add a model card or a because you can add a model card or a because you can add a model card or a data set card. We have tooling, so you data set card. We have tooling, so you data set card. We have tooling, so you can easily upload or download stuff from can easily upload or download stuff from can easily upload or download stuff from Hugging Face. Hugging Face. Hugging Face. And it might also help reach researchers And it might also help reach researchers And it might also help reach researchers in promoting their work. So, it's in promoting their work. So, it's in promoting their work. So, it's basically a win-win basically a win-win basically a win-win both for researchers and then other both for researchers and then other both for researchers and then other people using the research. people using the research. people using the research. So, yeah, these are the typical GitHub So, yeah, these are the typical GitHub So, yeah, these are the typical GitHub issues that I was opening. I always had issues that I was opening. I always had issues that I was opening. I always had like the same template. I just asked, like the same template. I just asked, like the same template. I just asked, "Could you please release this "Could you please release this "Could you please release this checkpoint on Hugging Face? Could you checkpoint on Hugging Face? Could you checkpoint on Hugging Face? Could you please release this data set on Hugging please release this data set on Hugging please release this data set on Hugging Face?" Face?" Face?" And [snorts] then I also opened PRs, And [snorts] then I also opened PRs, And [snorts] then I also opened PRs, pull requests on Hugging Face to add pull requests on Hugging Face to add pull requests on Hugging Face to add data set cards or model cards to improve data set cards or model cards to improve data set cards or model cards to improve the documentation of those artifacts. the documentation of those artifacts. the documentation of those artifacts. But, there's a problem. But, there's a problem. But, there's a problem. It's not really scalable for me to open It's not really scalable for me to open It's not really scalable for me to open all these GitHub issues or pull requests all these GitHub issues or pull requests all these GitHub issues or pull requests because every single day there are like because every single day there are like because every single day there are like hundreds of research papers coming out hundreds of research papers coming out hundreds of research papers coming out on archive, especially now with AI boom.

on archive, especially now with AI boom. on archive, especially now with AI boom. Um yeah, also NeurIPS, for example, a Um yeah, also NeurIPS, for example, a Um yeah, also NeurIPS, for example, a major AI conference, they are seeing a major AI conference, they are seeing a major AI conference, they are seeing a massive amount of papers. massive amount of papers. massive amount of papers. So, can we automate this? Can we scale So, can we automate this? Can we scale So, can we automate this? Can we scale the community science team with agents? the community science team with agents? the community science team with agents? So, that's the second part of my talk. So, that's the second part of my talk. So, that's the second part of my talk. How can we, yeah, scale this uh to a How can we, yeah, scale this uh to a How can we, yeah, scale this uh to a massive amount of research papers? massive amount of research papers? massive amount of research papers? So, the idea is pretty simple. Uh So, the idea is pretty simple. Uh So, the idea is pretty simple. Uh we should have an AI agent which can we should have an AI agent which can we should have an AI agent which can help me do this outreach to all these help me do this outreach to all these help me do this outreach to all these researchers which publish uh models or researchers which publish uh models or researchers which publish uh models or data sets uh as part of their research data sets uh as part of their research data sets uh as part of their research work. And then, yeah, do the outreach in work. And then, yeah, do the outreach in work. And then, yeah, do the outreach in an automated way. So, this is the an automated way. So, this is the an automated way. So, this is the typical workflow that I was following. typical workflow that I was following. typical workflow that I was following. So, basically, whenever I saw a research So, basically, whenever I saw a research So, basically, whenever I saw a research paper, I first tried to find the GitHub paper, I first tried to find the GitHub paper, I first tried to find the GitHub URL uh of that paper, if it's available. URL uh of that paper, if it's available. URL uh of that paper, if it's available. Then, I I read the readme of that GitHub Then, I I read the readme of that GitHub Then, I I read the readme of that GitHub file. And then, I basically check if file. And then, I basically check if file. And then, I basically check if there's anything new interesting to be there's anything new interesting to be there's anything new interesting to be shared on Hugging Face. shared on Hugging Face. shared on Hugging Face. Uh it could be that it's on Hugging Face Uh it could be that it's on Hugging Face Uh it could be that it's on Hugging Face already. In that case, I check whether already. In that case, I check whether already. In that case, I check whether the model cards or data set cards the model cards or data set cards the model cards or data set cards already properly already properly already properly uh present, whether the metadata tags, uh present, whether the metadata tags, uh present, whether the metadata tags, for example, are there.

for example, are there. for example, are there. If uh not, then I will might open a pull If uh not, then I will might open a pull If uh not, then I will might open a pull request. Otherwise, if the artifacts are request. Otherwise, if the artifacts are request. Otherwise, if the artifacts are not yet on Hugging Face, I open a GitHub not yet on Hugging Face, I open a GitHub not yet on Hugging Face, I open a GitHub issue. And then, finally, I also follow issue. And then, finally, I also follow issue. And then, finally, I also follow up with the author. So, that's kind of up with the author. So, that's kind of up with the author. So, that's kind of the workflow that I had to automate uh the workflow that I had to automate uh the workflow that I had to automate uh with agents. with agents. with agents. And there are several ways to solve And there are several ways to solve And there are several ways to solve this. Uh you could this. Uh you could this. Uh you could uh go with a workflow. uh go with a workflow. uh go with a workflow. Uh these pictures are, by the way, taken Uh these pictures are, by the way, taken Uh these pictures are, by the way, taken from the blog post building effective from the blog post building effective from the blog post building effective agents by Anthropic, which is a really agents by Anthropic, which is a really agents by Anthropic, which is a really great read. Uh read. Um so, on the left great read. Uh read. Um so, on the left great read. Uh read. Um so, on the left side, you see, yeah, a workflow which is side, you see, yeah, a workflow which is side, you see, yeah, a workflow which is more deterministic. You basically use more deterministic. You basically use more deterministic. You basically use LLM APIs within steps of a predefined LLM APIs within steps of a predefined LLM APIs within steps of a predefined path or pipeline, path or pipeline, path or pipeline, uh which is more predictable. It's more uh which is more predictable. It's more uh which is more predictable. It's more deterministic. You have more control deterministic. You have more control deterministic. You have more control over it. Of course, it's less flex- over it. Of course, it's less flex- over it. Of course, it's less flex- flexible. And then, on the other hand, flexible. And then, on the other hand, flexible. And then, on the other hand, you could have a fully fledged auto- you could have a fully fledged auto- you could have a fully fledged auto- autonomous agent, which is an LLM in a autonomous agent, which is an LLM in a autonomous agent, which is an LLM in a loop that calls tools until it's done, loop that calls tools until it's done, loop that calls tools until it's done, which is more flexible, but also less uh which is more flexible, but also less uh which is more flexible, but also less uh predictable. predictable. predictable. Uh Uh, at the time, yeah, of course, it Uh Uh, at the time, yeah, of course, it Uh Uh, at the time, yeah, of course, it doesn't have to be a binary story. You doesn't have to be a binary story. You doesn't have to be a binary story. You can have a workflow on one hand, you can can have a workflow on one hand, you can can have a workflow on one hand, you can have a fully autonomous agent on the have a fully autonomous agent on the have a fully autonomous agent on the other hand, but you you could of course other hand, but you you could of course other hand, but you you could of course also mix and match these type of things also mix and match these type of things also mix and match these type of things uh, for your use case.

uh, for your use case. uh, for your use case. In my case, I went for um, a pretty In my case, I went for um, a pretty In my case, I went for um, a pretty deterministic workflow. Uh, why? Because deterministic workflow. Uh, why? Because deterministic workflow. Uh, why? Because at the time that I was building this, at the time that I was building this, at the time that I was building this, this was in 2024, this was in 2024, this was in 2024, was at the time that Anthropic uh, wrote was at the time that Anthropic uh, wrote was at the time that Anthropic uh, wrote their blog post building effective their blog post building effective their blog post building effective agents. And there they actually said, agents. And there they actually said, agents. And there they actually said, "Try to avoid building agents if you "Try to avoid building agents if you "Try to avoid building agents if you really don't have to. Start simple, really don't have to. Start simple, really don't have to. Start simple, start with a single LLM API. start with a single LLM API. start with a single LLM API. Uh, avoid frameworks." Uh, and actually Uh, avoid frameworks." Uh, and actually Uh, avoid frameworks." Uh, and actually I think those were great tips. So, at I think those were great tips. So, at I think those were great tips. So, at the time I started building a workflow the time I started building a workflow the time I started building a workflow which basically replicated the workflow which basically replicated the workflow which basically replicated the workflow that I was doing when I was doing this that I was doing when I was doing this that I was doing when I was doing this outreach. outreach. outreach. So, yeah, this is the whole uh, So, yeah, this is the whole uh, So, yeah, this is the whole uh, pipeline. This is created using the pipeline. This is created using the pipeline. This is created using the Excalidraw MCP server in Cursor. It's Excalidraw MCP server in Cursor. It's Excalidraw MCP server in Cursor. It's pretty nice to create a visualization of pretty nice to create a visualization of pretty nice to create a visualization of your code. your code. your code. Uh, I'm not going to go into the Uh, I'm not going to go into the Uh, I'm not going to go into the details, but basically it just details, but basically it just details, but basically it just replicates um, the workflow that I was replicates um, the workflow that I was replicates um, the workflow that I was doing when doing the outreach. And I use doing when doing the outreach. And I use doing when doing the outreach. And I use LLM APIs in then each of the steps LLM APIs in then each of the steps LLM APIs in then each of the steps without any framework, without any agent without any framework, without any agent without any framework, without any agent framework. So, it made it quite uh, framework. So, it made it quite uh, framework. So, it made it quite uh, deterministic and I had a lot of control deterministic and I had a lot of control deterministic and I had a lot of control over uh, how this goes.

over uh, how this goes. over uh, how this goes. Um, in terms of deployment of this uh, Um, in terms of deployment of this uh, Um, in terms of deployment of this uh, workflow, it's a simple cron job. So, workflow, it's a simple cron job. So, workflow, it's a simple cron job. So, cron is just something that runs cron is just something that runs cron is just something that runs regularly. In my case, I run it once regularly. In my case, I run it once regularly. In my case, I run it once every night. So, when I'm sleeping, every night. So, when I'm sleeping, every night. So, when I'm sleeping, there is this agent, but technically there is this agent, but technically there is this agent, but technically it's just a cron job, a Python script it's just a cron job, a Python script it's just a cron job, a Python script with an LLM API, which is going to read with an LLM API, which is going to read with an LLM API, which is going to read all these hundreds of archive papers, all these hundreds of archive papers, all these hundreds of archive papers, uh, and then it might open GitHub issues uh, and then it might open GitHub issues uh, and then it might open GitHub issues or it might open pull requests on or it might open pull requests on or it might open pull requests on Hugging Face. Hugging Face. Hugging Face. I'm using GitHub Actions for this. Uh, I I'm using GitHub Actions for this. Uh, I I'm using GitHub Actions for this. Uh, I saw this very nice blog post free cron saw this very nice blog post free cron saw this very nice blog post free cron jobs with GitHub Actions, and actually jobs with GitHub Actions, and actually jobs with GitHub Actions, and actually it's probably the best entry point if it's probably the best entry point if it's probably the best entry point if you want to set up cron jobs, you want to set up cron jobs, you want to set up cron jobs, um, because GitHub has a pretty generous um, because GitHub has a pretty generous um, because GitHub has a pretty generous tier if you want to get started with tier if you want to get started with tier if you want to get started with like putting simple cron jobs uh, up like putting simple cron jobs uh, up like putting simple cron jobs uh, up there. And yeah, it makes it really easy there. And yeah, it makes it really easy there. And yeah, it makes it really easy for me in the UI to manage all these for me in the UI to manage all these for me in the UI to manage all these cron jobs. cron jobs. cron jobs. So yeah, every night I have hundreds of So yeah, every night I have hundreds of So yeah, every night I have hundreds of uh uh uh GitHub issues being created. GitHub issues being created. GitHub issues being created. For the tracing part, For the tracing part, For the tracing part, um I'm using LangFuse. Uh yeah, LangFuse um I'm using LangFuse. Uh yeah, LangFuse um I'm using LangFuse. Uh yeah, LangFuse also has a a booth here. also has a a booth here. also has a a booth here. Um LangFuse is pretty great.

Um LangFuse is pretty great. Um LangFuse is pretty great. Um I use it mostly for the tracing part, Um I use it mostly for the tracing part, Um I use it mostly for the tracing part, the observability part, just to see what the observability part, just to see what the observability part, just to see what is the LLM doing, what are the inputs, is the LLM doing, what are the inputs, is the LLM doing, what are the inputs, what are the outputs, what are the outputs, what are the outputs, what are the prompts, how much does it what are the prompts, how much does it what are the prompts, how much does it cost, latency, and so on. cost, latency, and so on. cost, latency, and so on. Um so yeah, uh I definitely recommend Um so yeah, uh I definitely recommend Um so yeah, uh I definitely recommend it. it. it. Um but yeah, as my agents are opening so Um but yeah, as my agents are opening so Um but yeah, as my agents are opening so many GitHub issues every night, I then many GitHub issues every night, I then many GitHub issues every night, I then end up with a massive amount of unread end up with a massive amount of unread end up with a massive amount of unread GitHub notifications because people GitHub notifications because people GitHub notifications because people reply to those GitHub issues. reply to those GitHub issues. reply to those GitHub issues. And that's a lot of work to then reply And that's a lot of work to then reply And that's a lot of work to then reply to all of those issues. It's kind of to all of those issues. It's kind of to all of those issues. It's kind of like going through your mailbox. like going through your mailbox. like going through your mailbox. So you could wonder, could we also um So you could wonder, could we also um So you could wonder, could we also um automate the follow-up to those GitHub automate the follow-up to those GitHub automate the follow-up to those GitHub uh issues? Because initially I was still uh issues? Because initially I was still uh issues? Because initially I was still the GitHub issue creation was done uh by the GitHub issue creation was done uh by the GitHub issue creation was done uh by an agent, but I was still the one an agent, but I was still the one an agent, but I was still the one involved in then doing the follow-up. involved in then doing the follow-up. involved in then doing the follow-up. Uh now a few months ago I also automated Uh now a few months ago I also automated Uh now a few months ago I also automated the the follow-up to those GitHub the the follow-up to those GitHub the the follow-up to those GitHub issues. issues. issues. Again, you could think, how should you Again, you could think, how should you Again, you could think, how should you solve this? Should you go for a more solve this? Should you go for a more solve this? Should you go for a more deterministic workflow or can you go for deterministic workflow or can you go for deterministic workflow or can you go for a fully autonomous agents, uh an LLM in a fully autonomous agents, uh an LLM in a fully autonomous agents, uh an LLM in a loop which runs with some tools and a loop which runs with some tools and a loop which runs with some tools and skills?

skills? skills? Um well, here I went for kind of a fully Um well, here I went for kind of a fully Um well, here I went for kind of a fully autonomous agents, uh so it's kind of autonomous agents, uh so it's kind of autonomous agents, uh so it's kind of flexible. It's a bit less predictable, flexible. It's a bit less predictable, flexible. It's a bit less predictable, but it works quite well. Um I went for but it works quite well. Um I went for but it works quite well. Um I went for this because uh in November of last year this because uh in November of last year this because uh in November of last year at AI Engineer in New York, there was a at AI Engineer in New York, there was a at AI Engineer in New York, there was a pretty nice workshop by Anthropic on the pretty nice workshop by Anthropic on the pretty nice workshop by Anthropic on the Claude agent SDK. Claude agent SDK. Claude agent SDK. And there they were actually saying that And there they were actually saying that And there they were actually saying that agents might be better than workflows. agents might be better than workflows. agents might be better than workflows. So they So they were kind of So they So they were kind of So they So they were kind of contradicting themselves, but they he contradicting themselves, but they he contradicting themselves, but they he said that models have become so good said that models have become so good said that models have become so good that you might actually now start to that you might actually now start to that you might actually now start to work with fully autonomous agents rather work with fully autonomous agents rather work with fully autonomous agents rather than a workflow. than a workflow. than a workflow. So this is why I went with this approach So this is why I went with this approach So this is why I went with this approach and I actually am using the Claude and I actually am using the Claude and I actually am using the Claude agents SDK for this use case. agents SDK for this use case. agents SDK for this use case. Uh there was another pretty nice talk by Uh there was another pretty nice talk by Uh there was another pretty nice talk by Cursor also at AI Engineer. This was in Cursor also at AI Engineer. This was in Cursor also at AI Engineer. This was in the European version in London a few the European version in London a few the European version in London a few months ago. months ago. months ago. There they talked about how they There they talked about how they There they talked about how they replaced 12,000 lines of custom code, replaced 12,000 lines of custom code, replaced 12,000 lines of custom code, pretty sophisticated workflow, with a pretty sophisticated workflow, with a pretty sophisticated workflow, with a very simple 200 lines of code skill. very simple 200 lines of code skill. very simple 200 lines of code skill. Uh actually it's pretty similar for me Uh actually it's pretty similar for me Uh actually it's pretty similar for me like I can like I can like I can replace a lot of custom codes, thousands replace a lot of custom codes, thousands replace a lot of custom codes, thousands of lines of code, with nowadays just a of lines of code, with nowadays just a of lines of code, with nowadays just a simple agent with maybe a CLI as a tool simple agent with maybe a CLI as a tool simple agent with maybe a CLI as a tool and a skill and that's it and a skill and that's it and a skill and that's it cuz the models have become so good.

cuz the models have become so good. cuz the models have become so good. So yeah, in terms of the So yeah, in terms of the So yeah, in terms of the architecture, this is a bit what it architecture, this is a bit what it architecture, this is a bit what it looks like. looks like. looks like. Um so it's actually just the Claude Um so it's actually just the Claude Um so it's actually just the Claude agents SDK which is, I would say, a agents SDK which is, I would say, a agents SDK which is, I would say, a pretty good Python SDK for building an pretty good Python SDK for building an pretty good Python SDK for building an agent. agent. agent. Initially I was using the Claude models, Initially I was using the Claude models, Initially I was using the Claude models, but then I since actually this week I'm but then I since actually this week I'm but then I since actually this week I'm using the GLM 5.2 model via Hugging Face using the GLM 5.2 model via Hugging Face using the GLM 5.2 model via Hugging Face inference providers. So Hugging Face inference providers. So Hugging Face inference providers. So Hugging Face does offer does offer does offer a service which basically wraps a lot of a service which basically wraps a lot of a service which basically wraps a lot of inference providers like Together AI, inference providers like Together AI, inference providers like Together AI, Fireworks, Cerebras and so on. So you Fireworks, Cerebras and so on. So you Fireworks, Cerebras and so on. So you can use a lot of open models in a can use a lot of open models in a can use a lot of open models in a unified way. It's OpenAI compatible unified way. It's OpenAI compatible unified way. It's OpenAI compatible or Anthropic compatible and then I or Anthropic compatible and then I or Anthropic compatible and then I deploy this on Modal. Modal is also deploy this on Modal. Modal is also deploy this on Modal. Modal is also present here today. present here today. present here today. And it's mainly using Bash as a tool so And it's mainly using Bash as a tool so And it's mainly using Bash as a tool so the terminal to basically the terminal to basically the terminal to basically do Hugging Face commands because it's do Hugging Face commands because it's do Hugging Face commands because it's using the Hugging Face CLI quite a bit. using the Hugging Face CLI quite a bit. using the Hugging Face CLI quite a bit. So I combine it with the Hugging Face So I combine it with the Hugging Face So I combine it with the Hugging Face CLI skill which is actually all it CLI skill which is actually all it CLI skill which is actually all it needs. And then needs. And then needs. And then it might comment something on GitHub as it might comment something on GitHub as it might comment something on GitHub as a follow-up. And it also actually does a follow-up. And it also actually does a follow-up. And it also actually does the posting on Slack because eventually the posting on Slack because eventually the posting on Slack because eventually I also want to see the final results on I also want to see the final results on I also want to see the final results on our Slack channel our Slack channel our Slack channel uh from Hugging Face. So yeah, given uh from Hugging Face. So yeah, given uh from Hugging Face. So yeah, given that there's also a lot of hype on GLM that there's also a lot of hype on GLM that there's also a lot of hype on GLM 5.2 recently, for example, Cursor uh saw 5.2 recently, for example, Cursor uh saw 5.2 recently, for example, Cursor uh saw great performance on their Cursor bench.

great performance on their Cursor bench. great performance on their Cursor bench. Post-training bench is another one uh Post-training bench is another one uh Post-training bench is another one uh where it actually beats Opus 4.8 and where it actually beats Opus 4.8 and where it actually beats Opus 4.8 and it's cheaper. So, it's cheaper. So, it's cheaper. So, yeah, there's no reason not to use GLM yeah, there's no reason not to use GLM yeah, there's no reason not to use GLM 5.2 uh especially given that I work at 5.2 uh especially given that I work at 5.2 uh especially given that I work at Hugging Face. Hugging Face. Hugging Face. Um for the deployment, as I said before, Um for the deployment, as I said before, Um for the deployment, as I said before, I use Modal. I use Modal. I use Modal. It's pretty great if you want to deploy It's pretty great if you want to deploy It's pretty great if you want to deploy agents. Uh in my case, I'm using the agents. Uh in my case, I'm using the agents. Uh in my case, I'm using the batch processing feature. So, they allow batch processing feature. So, they allow batch processing feature. So, they allow you to spin up a massive amount of you to spin up a massive amount of you to spin up a massive amount of containers all in parallel. Every single containers all in parallel. Every single containers all in parallel. Every single container is basically one agent loop container is basically one agent loop container is basically one agent loop that is processing one GitHub issue. that is processing one GitHub issue. that is processing one GitHub issue. Uh it's super easy to use, I have to Uh it's super easy to use, I have to Uh it's super easy to use, I have to say. say. say. Um Um Um and the startups are also pretty fast. and the startups are also pretty fast. and the startups are also pretty fast. So, I definitely recommend it if you're So, I definitely recommend it if you're So, I definitely recommend it if you're building uh agents that are like, for building uh agents that are like, for building uh agents that are like, for example, running in the background, example, running in the background, example, running in the background, running overnight, for example. running overnight, for example. running overnight, for example. Um and then the way I invoke it, yeah, Um and then the way I invoke it, yeah, Um and then the way I invoke it, yeah, technically I could also just uh deploy technically I could also just uh deploy technically I could also just uh deploy this as a cron job. Modal, for example, this as a cron job. Modal, for example, this as a cron job. Modal, for example, has support for this. But typically, the has support for this. But typically, the has support for this. But typically, the follow-up on the GitHub issues, I still follow-up on the GitHub issues, I still follow-up on the GitHub issues, I still do that actually manually by invoking it do that actually manually by invoking it do that actually manually by invoking it as a skill. So, I created a skill for as a skill. So, I created a skill for as a skill. So, I created a skill for this in Cursor.

this in Cursor. this in Cursor. Uh I call it process under it Modal. And Uh I call it process under it Modal. And Uh I call it process under it Modal. And then what it's going to do is it's then what it's going to do is it's then what it's going to do is it's actually going to invoke an agent, in actually going to invoke an agent, in actually going to invoke an agent, in this case, Composer 2.5, which is like this case, Composer 2.5, which is like this case, Composer 2.5, which is like the agent that I'm mostly using in the agent that I'm mostly using in the agent that I'm mostly using in Cursor, which is again going to invoke Cursor, which is again going to invoke Cursor, which is again going to invoke all the other agents. So, that's this is all the other agents. So, that's this is all the other agents. So, that's this is kind of the loop that people are talking kind of the loop that people are talking kind of the loop that people are talking about. And then finally, it's going to about. And then finally, it's going to about. And then finally, it's going to post uh all the results on our Slack post uh all the results on our Slack post uh all the results on our Slack channel. channel. channel. Uh so yeah, and this is actually what it Uh so yeah, and this is actually what it Uh so yeah, and this is actually what it just posts. So, what it does is it just posts. So, what it does is it just posts. So, what it does is it basically just posts a huge amount of basically just posts a huge amount of basically just posts a huge amount of Hugging Face papers, Hugging Face papers, Hugging Face papers, uh uh uh which are these research papers which which are these research papers which which are these research papers which people can uh make available on Hugging people can uh make available on Hugging people can uh make available on Hugging Face because every time someone mentions Face because every time someone mentions Face because every time someone mentions it in a model card or dataset card, we it in a model card or dataset card, we it in a model card or dataset card, we index it on the hub. And then it just index it on the hub. And then it just index it on the hub. And then it just posts all the artifacts that people have posts all the artifacts that people have posts all the artifacts that people have been uploading based on the outreach been uploading based on the outreach been uploading based on the outreach that we do via GitHub. that we do via GitHub. that we do via GitHub. Um so yeah, I do this still in a manual Um so yeah, I do this still in a manual Um so yeah, I do this still in a manual form. So I just invoke the skill and form. So I just invoke the skill and form. So I just invoke the skill and then after a few minutes, these messages then after a few minutes, these messages then after a few minutes, these messages appear on our Slack channel. appear on our Slack channel. appear on our Slack channel. Um yeah, I just included some fun Um yeah, I just included some fun Um yeah, I just included some fun results because to be honest, it's quite results because to be honest, it's quite results because to be honest, it's quite fun to see people interacting with the fun to see people interacting with the fun to see people interacting with the agents.

agents. agents. Um Um Um to be honest, I don't disclose that it's to be honest, I don't disclose that it's to be honest, I don't disclose that it's an agent. Why? Because I think if people an agent. Why? Because I think if people an agent. Why? Because I think if people know it's a bot, then they might quickly know it's a bot, then they might quickly know it's a bot, then they might quickly like close the issue. And to be honest, like close the issue. And to be honest, like close the issue. And to be honest, they post exactly the same stuff as I they post exactly the same stuff as I they post exactly the same stuff as I was doing before manually. So I don't was doing before manually. So I don't was doing before manually. So I don't actually see any reason to to do that. actually see any reason to to do that. actually see any reason to to do that. Um so and then you see replies like Um so and then you see replies like Um so and then you see replies like this. Hi Niels, thanks a lot for your this. Hi Niels, thanks a lot for your this. Hi Niels, thanks a lot for your suggestion and the clear guidance. suggestion and the clear guidance. suggestion and the clear guidance. I actually also often times see people I actually also often times see people I actually also often times see people using an agent to reply to my agents. So using an agent to reply to my agents. So using an agent to reply to my agents. So it's kind of the that internet nowadays. it's kind of the that internet nowadays. it's kind of the that internet nowadays. Um but people yeah, make all their Um but people yeah, make all their Um but people yeah, make all their artifacts available on Hugging Face. And artifacts available on Hugging Face. And artifacts available on Hugging Face. And out of the thousands of issues that are out of the thousands of issues that are out of the thousands of issues that are being created on Hugging Face, actually being created on Hugging Face, actually being created on Hugging Face, actually so far I've only had two negative so far I've only had two negative so far I've only had two negative comments. One guy saying yeah, please comments. One guy saying yeah, please comments. One guy saying yeah, please close this slop. So he closed the issue. close this slop. So he closed the issue. close this slop. So he closed the issue. And then another one. But most of the And then another one. But most of the And then another one. But most of the people they just say, yeah, actually it people they just say, yeah, actually it people they just say, yeah, actually it makes perfect sense to make my weights makes perfect sense to make my weights makes perfect sense to make my weights or my data sets available on Hugging or my data sets available on Hugging or my data sets available on Hugging Face. Like why didn't I think of this? Face. Like why didn't I think of this? Face. Like why didn't I think of this? Um so it's kind of a win-win I would Um so it's kind of a win-win I would Um so it's kind of a win-win I would say. say. say. Uh I often times Uh I often times Uh I often times also post fun results on our Slack also post fun results on our Slack also post fun results on our Slack channel. Like for example, one time channel. Like for example, one time channel. Like for example, one time someone a researcher from Apple someone a researcher from Apple someone a researcher from Apple sent me a DM like, I saw you reached out sent me a DM like, I saw you reached out sent me a DM like, I saw you reached out to me. Yeah, technically it's my agent to me. Yeah, technically it's my agent to me. Yeah, technically it's my agent just posting a GitHub issue regarding just posting a GitHub issue regarding just posting a GitHub issue regarding publishing a new Apple publishing a new Apple publishing a new Apple uh the artifacts of an Apple paper on uh the artifacts of an Apple paper on uh the artifacts of an Apple paper on Hugging Face. Or for example, it reaches Hugging Face. Or for example, it reaches Hugging Face. Or for example, it reaches out to Google DeepMind out to Google DeepMind out to Google DeepMind to publish to publish to publish um mathematics data sets.

um mathematics data sets. um mathematics data sets. Um so a lot of times like I receive Um so a lot of times like I receive Um so a lot of times like I receive emails, the one on the the side, where emails, the one on the the side, where emails, the one on the the side, where yeah, they want to publish a 400 GB yeah, they want to publish a 400 GB yeah, they want to publish a 400 GB data set on Hugging Face, but this was data set on Hugging Face, but this was data set on Hugging Face, but this was also my agent just opening GitHub also my agent just opening GitHub also my agent just opening GitHub issues. issues. issues. Um Um Um Yeah, this is another fun result. So, Yeah, this is another fun result. So, Yeah, this is another fun result. So, Paddle OCR, it's like a Chinese company. Paddle OCR, it's like a Chinese company. Paddle OCR, it's like a Chinese company. They migrated all their OCR models to They migrated all their OCR models to They migrated all their OCR models to Hugging Face based on outreach by the Hugging Face based on outreach by the Hugging Face based on outreach by the agents that create issues for me. So, agents that create issues for me. So, agents that create issues for me. So, yeah, it's pretty nice. yeah, it's pretty nice. yeah, it's pretty nice. Another fun result is like when it when Another fun result is like when it when Another fun result is like when it when it completes the default template of it completes the default template of it completes the default template of model cards on Hugging Face. So, Mac model cards on Hugging Face. So, Mac model cards on Hugging Face. So, Mac Mitchell, who also works at Hugging Mitchell, who also works at Hugging Mitchell, who also works at Hugging Face, she has a famous paper called Face, she has a famous paper called Face, she has a famous paper called model cards for model reporting, making model cards for model reporting, making model cards for model reporting, making sure that anyone documents their models sure that anyone documents their models sure that anyone documents their models in a proper way. And so, we do provide in a proper way. And so, we do provide in a proper way. And so, we do provide this template, which you can see on the this template, which you can see on the this template, which you can see on the left side in the Git diff. And then, the left side in the Git diff. And then, the left side in the Git diff. And then, the agent is just completing that template agent is just completing that template agent is just completing that template based on the content that it finds based based on the content that it finds based based on the content that it finds based on the paper, like the GitHub readme, on the paper, like the GitHub readme, on the paper, like the GitHub readme, the PDF itself, and so on.

Um yeah, it's also quite funny to see, Um yeah, it's also quite funny to see, for example, in this case that it for example, in this case that it for example, in this case that it included me in included me in included me in the model card. It said, "Model card the model card. It said, "Model card the model card. It said, "Model card authors, Niels part of the Hugging Face authors, Niels part of the Hugging Face authors, Niels part of the Hugging Face community science team." I never community science team." I never community science team." I never prompted it this way, but it's pretty prompted it this way, but it's pretty prompted it this way, but it's pretty fun to see. fun to see. fun to see. Or people are replying, "Thank you for Or people are replying, "Thank you for Or people are replying, "Thank you for helping me fix my mistakes." So, those helping me fix my mistakes." So, those helping me fix my mistakes." So, those are all done by are all done by are all done by the agents. Uh I think the most popular GitHub issue Uh I think the most popular GitHub issue that was created was this paper Tiny that was created was this paper Tiny that was created was this paper Tiny Recursive Models, which you might have Recursive Models, which you might have Recursive Models, which you might have seen, was quite trending seen, was quite trending seen, was quite trending both on Hugging Face, but also on both on Hugging Face, but also on both on Hugging Face, but also on Twitter. Twitter. Twitter. So, yeah, more than 60 people actually So, yeah, more than 60 people actually So, yeah, more than 60 people actually upvoted that issue so that the model was upvoted that issue so that the model was upvoted that issue so that the model was released on Hugging Face. So, this is released on Hugging Face. So, this is released on Hugging Face. So, this is again, I think, the win-win. So, it's again, I think, the win-win. So, it's again, I think, the win-win. So, it's both a win for the researcher, making both a win for the researcher, making both a win for the researcher, making their research more discoverable on their research more discoverable on their research more discoverable on Hugging Face, Hugging Face, Hugging Face, but it's also, yeah, better for the but it's also, yeah, better for the but it's also, yeah, better for the people then who want to build on top of people then who want to build on top of people then who want to build on top of that research and want to that research and want to that research and want to use them. use them. use them. Uh so, yeah, I I hundreds of GitHub Uh so, yeah, I I hundreds of GitHub Uh so, yeah, I I hundreds of GitHub issues where I think I can show uh nice issues where I think I can show uh nice issues where I think I can show uh nice results results results um um um where people interact with the agents.

where people interact with the agents. where people interact with the agents. You might also wonder, yeah, how to You might also wonder, yeah, how to You might also wonder, yeah, how to avoid slop because you might think, avoid slop because you might think, avoid slop because you might think, okay, you have an agent uh spamming the okay, you have an agent uh spamming the okay, you have an agent uh spamming the whole internet with your GitHub issues. whole internet with your GitHub issues. whole internet with your GitHub issues. Like, should you even do this? Again, I Like, should you even do this? Again, I Like, should you even do this? Again, I already talked about the win-win. Um, already talked about the win-win. Um, already talked about the win-win. Um, but a blog post that I highly recommend, but a blog post that I highly recommend, but a blog post that I highly recommend, if you want to avoid that your agent is if you want to avoid that your agent is if you want to avoid that your agent is just posting slop, is um the LLM Evils just posting slop, is um the LLM Evils just posting slop, is um the LLM Evils FAQ FAQ FAQ uh by Hamel Husain. Uh, I would say he's uh by Hamel Husain. Uh, I would say he's uh by Hamel Husain. Uh, I would say he's like the main expert when it comes to like the main expert when it comes to like the main expert when it comes to LLM evaluation. LLM evaluation. LLM evaluation. He also has like a a paid course, but he He also has like a a paid course, but he He also has like a a paid course, but he also publishes a lot of stuff for free also publishes a lot of stuff for free also publishes a lot of stuff for free online including this blog post. So, I online including this blog post. So, I online including this blog post. So, I highly recommend to go through it if you highly recommend to go through it if you highly recommend to go through it if you want to learn more about how to evaluate want to learn more about how to evaluate want to learn more about how to evaluate your agents. your agents. your agents. So, my conclusion would be um that open So, my conclusion would be um that open So, my conclusion would be um that open models are actually getting great, models are actually getting great, models are actually getting great, especially now with GLM 5.2. You have especially now with GLM 5.2. You have especially now with GLM 5.2. You have Deep Seek V4 and so on. So, um yeah, we Deep Seek V4 and so on. So, um yeah, we Deep Seek V4 and so on. So, um yeah, we we are able to now replace closed-source we are able to now replace closed-source we are able to now replace closed-source models by open ones. models by open ones. models by open ones. Uh, for my use case, I would say agents Uh, for my use case, I would say agents Uh, for my use case, I would say agents are actually better than uh workflows. are actually better than uh workflows. are actually better than uh workflows. Uh, they only need a single CLI, which Uh, they only need a single CLI, which Uh, they only need a single CLI, which is the Hugging Face CLI. They need a is the Hugging Face CLI. They need a is the Hugging Face CLI. They need a single skill, the Hugging Face CLI single skill, the Hugging Face CLI single skill, the Hugging Face CLI skill, and a sandbox, and that's all skill, and a sandbox, and that's all skill, and a sandbox, and that's all they need to do their work.

they need to do their work. they need to do their work. And finally, yeah, don't forget about And finally, yeah, don't forget about And finally, yeah, don't forget about evaluation. evaluation. evaluation. Um Finally, uh I can also discuss some Finally, uh I can also discuss some other efforts that we do as part of the other efforts that we do as part of the other efforts that we do as part of the community science uh team. community science uh team. community science uh team. Um, very shortly. Um, so, I have a Um, very shortly. Um, so, I have a Um, very shortly. Um, so, I have a Twitter account that I created. It's Twitter account that I created. It's Twitter account that I created. It's called Daily Papers. called Daily Papers. called Daily Papers. And it actually uses the exact same And it actually uses the exact same And it actually uses the exact same workflow as my agents behind the scenes workflow as my agents behind the scenes workflow as my agents behind the scenes to post interesting research papers on to post interesting research papers on to post interesting research papers on X. X. X. It uh recently crossed 90,000 followers It uh recently crossed 90,000 followers It uh recently crossed 90,000 followers without any involvement of me. I just without any involvement of me. I just without any involvement of me. I just deployed this uh and it posts deployed this uh and it posts deployed this uh and it posts interesting research papers and interesting research papers and interesting research papers and artifacts from Hugging Face every 4 artifacts from Hugging Face every 4 artifacts from Hugging Face every 4 hours or every time someone uh releases hours or every time someone uh releases hours or every time someone uh releases something cool on the Hugging Face. something cool on the Hugging Face. something cool on the Hugging Face. Um so, yeah. Um so, yeah. Um so, yeah. And I have like Gemini determining the And I have like Gemini determining the And I have like Gemini determining the best visual to tweet or to include in best visual to tweet or to include in best visual to tweet or to include in the tweet. Like for example, this recent the tweet. Like for example, this recent the tweet. Like for example, this recent tweet tweet tweet where it tweeted out that Nvidia where it tweeted out that Nvidia where it tweeted out that Nvidia released an optimized version of GLM 5.2 released an optimized version of GLM 5.2 released an optimized version of GLM 5.2 got more than 2,000 likes. So, that's got more than 2,000 likes. So, that's got more than 2,000 likes. So, that's pretty cool to see. pretty cool to see. pretty cool to see. And a final effort that I'm working on And a final effort that I'm working on And a final effort that I'm working on right now is a revival of Papers With right now is a revival of Papers With right now is a revival of Papers With Code, which is a website that once Code, which is a website that once Code, which is a website that once existed and then was acquired by Meta existed and then was acquired by Meta existed and then was acquired by Meta and then sadly it died. So, I'm I'm and then sadly it died. So, I'm I'm and then sadly it died. So, I'm I'm trying to re- revive it and making a trying to re- revive it and making a trying to re- revive it and making a research and state-of-the-art easier research and state-of-the-art easier research and state-of-the-art easier accessible. Um for now it lives at accessible. Um for now it lives at accessible. Um for now it lives at paperswithcode.co.

Uh so, yeah. You can find benchmarks Uh so, yeah. You can find benchmarks over there. For example, for OCR models, over there. For example, for OCR models, over there. For example, for OCR models, all OCR benches like popular benchmark. all OCR benches like popular benchmark. all OCR benches like popular benchmark. But I'm also making it an educational But I'm also making it an educational But I'm also making it an educational resource so that people can learn about resource so that people can learn about resource so that people can learn about technical terms like mixed training technical terms like mixed training technical terms like mixed training uh on policy distillation and so on. uh on policy distillation and so on. uh on policy distillation and so on. So, yeah. That was it for my talk. I So, yeah. That was it for my talk. I So, yeah. That was it for my talk. I hope you learned something. Thanks all hope you learned something. Thanks all hope you learned something. Thanks all of you for your attention. of you for your attention. of you for your attention. >> [applause]