Why Compute Still Trades in Group Chats — Affaan Mustafa & Alejandro Soumah Fernandez, Jump Trading

[music] >> Um hey everyone. My name is Afaan and I'm I'm a lead for compute trading at Jump Trading, which is a proprietary trading firm, alongside my co-lead over here Alejandro. So, today we're going to be looking over some of the market structure as it applies to compute trading, what compute trading even is, um what we can see it becoming. And I think we'll start with some macro, look at micro structure, and then talk a little bit about where we see things in policy. So, the title of the talk is the dark pools of compute, but some of you guys are in finance, so you might be confused as to why I'm using this terminology. Uh what do I mean by dark pool here? Essentially, what I'm referring to is an unlit market. Um right now, most compute trades in blocks, physical compute here, and these are private quotes. We have a lot of brokers and these are long bilateral contracts. Now, at the end of the day, these are really commercial agreements. So, we're using the term trading here to kind of like look at how it's transacted between parties. These commercial agreements tend to be some mixture of an MSA, an SLA, and an order form, cluster specific uh with hardware specific language in each kind of um in each order. And but when we're looking at pricing and looking at some

of these other kind of uh metrics, a lot of the time, most of the talk is around the skew. So, what the chip itself is, the base board. So, H100, H200, B300, uh etc. And so, um a lot of the negotiation here happens kind of outside the public view. There's no, you know, like exchange for physically traded compute that's widely, you know, adopted and you know, used for everyone goes there to like take a look and trade uh compute. These are done in chats, right? They're out of the public view. Um there's public marketplaces, there's price references, there's a lot of um you know, on-demand marketplaces. However, for these actual tenured, so you know, on a given term for a fixed amount of capacity uh with a certain total contract value, these trade over Slack messages, X messages, Telegram group chats. It's a bunch of OTC kind of channels that are all over in these kind of I want to call them like neo neo chat. Um and some of you might be familiar with commodities and how a lot of you know, there's chatting happening on Ice Chat, Bloomberg Chat, etc. This is kind of where where the where the kind of markets here lie.

And so I want to look at a little bit here on the macro market structure and a little bit on the players here as well. And so this was a recent kind of a16z, some of you guys might have seen this. So I'm I'm putting this out there because you know, you can go on a16z's actual X and take a look at this graph yourself. What we're looking at is that the cost of a given capability has fallen sharply. The bill of course still depends on the model usage, capacity, and other factors such as speed is a big thing right now with the new code ChatGPT models. Um cheaper tokens unlock more usage and a team can do more work for the same budget. So what we're seeing is that a lot of teams are simply just using way more tokens. So we have this latent demand that hasn't really been hit yet and people were just going to use more with the same budget that they were using before. One model's cost more and they're just expending more tokens. Um so applications that were too expensive become possible. Um however, as we're kind of talking about compute here, the capacity underneath still has to be built and paid for um even as the cost of a particular task comes down. And so the important distinction is between the price of the intelligence, the number of tasks people want to run, and the amount of hardware needed to serve them.

And so this is where we get to kind of the Jevons paradox and a little of some of the stuff that we've been looking at lately where we had a very interesting thing where, you know, intuitively you'd assume that compute and inference kind of like rises together, correlates very, you know, very closely as you can see. You can see there's a little bit of a decoupling that happened and token prices declined further but compute prices have remained robust. And so like what what can be an explanation for this? Now, there's a lot of like potential explanations. One here is the whole Jevons paradox is that even though token prices and inference is getting extremely efficient to serve on these new baseboards and and and and GPUs, people are just consuming a lot more. And so the demand for compute rises and people just consume a lot more tokens even if the cost per cost per million for input and for caching and, you know, for output becomes a lot more efficient. And so making the task cheaper, encouraging the use that way resource saving per task. For inference clouds, that's going to be more applications, serving more tokens, bigger market share.

And for app layer individuals, that means you guys can serve to a lot more kind of customers. So anyone who's building utilizing the token infrastructure. And so what really goes into, you know, the macro structure. Now like this is very, you know, bare bones here but you've got your chips which we've been talking about. You have power, you have buildings, grid connections and then of course you need a lot of capital to get these build outs going and then that's kind of been the whole talk here. And so um chips powers buildings money they all have to come together before, you know, you can even get to an hour of like actual usable GPU per hour compute. And so the IEA April update projects that the global global data center electricity is going to rise from 485 terawatt hours in 2025 to 915 2030. And so a lot of the talk right now is around build-outs, about getting the power grid set up. Will we have enough power grid here? Is there other energy sources that we can tap into? Where are we, you know, able to get the land to actually be able to do this? And so, colocation sites and data center operators, and people who are doing the financing the build-outs or the lenders themselves, are all kind of looking into this. And that type of everything that's upstream, essentially, of the compute is going to then flow down into the compute uh costs and also usage.

And so, what does this like actual macro map look like underneath a quote chip and system vendors supply the GPU servers and networking, sites and operators supply the power, space assembly and management right here, GPUs memory servers and networking, um capacity providers, these are hyperscalers and neo clouds and any other operator that's like operating similar to how a neo cloud would operate, reselling bare metal capacity, and then the users. So, who's the actual end user? The startups, the inference clouds uh labs, neo labs, and also anyone on the app layer and enterprises for anyone doing open weight stuff as well. And um these boxes, like I've kind of boxed them out here, but, you know, there's people can be all of the above. A chip and system vendor, like someone could do a full end-to-end build-out and also be the end user of everything. And so, there's a little bit of um variation that that that occurs here. And so, as I was kind of mentioning about the inference clouds before, they might rent a cluster and they'll serve tokens off of it and they'll use that revenue to the pay off the cluster. Um and so, that might be a use case for the actual inference cloud themselves.

However, for hyperscalers, frontier labs, and larger neo clouds, they can make substantial commitments because they have the ability to get into the large equity, they have the ability to take on more debt. And uh they have the ability to do long horizon kind of deals. So, 10-year deal opening I contracted it out, etc. So, of that nature. However, anyone in the middle market isn't actually able to do this. Who's underwriting a Series B startup with enough capacity to get in a 200 million plus contract over 3 years for compute? How many equity rounds will they have to raise, right? Will they have to use their entire equity round to fund just the actual cluster and then they don't have any remaining capital left to use for actual costs? And so, we're looking at a little bit of the financing chain here and how that plays in. So, lenders are financing the buildout against contracted revenue even in many cases for inference clouds. And so, they want to finance the buildout. They want confidence that their loan will get repaid. I mean, like how are you going to underwrite a startup like a private company that doesn't have like much else going for them right now. Like what are the, you know, they're not publicly traded. There's not that much cash flows, balance sheet, income statement. They might even be pre-profit, pre-revenue, etc. And so, what they're looking at is that can the person get contracted revenue over a certain scale. And so, Coreweave's public filings give large-scale example infrastructure financing supported by take or pay customer contracts. And so, this diagram kind of shows the smaller illustrative example here. And the mismatch is that hardware financing they want very long terms, but customers are only able to

afford or don't really want to like get into a long-term financing over 36 months. They have to put up on up to 15 to 25% is what Coreweave has quoted that they've seen their median prepayment to be, 15 to 25% up front on a 100 million TCV cluster. That's 15 to 25 million dollars that a startup has to put up front. That might be their entire round. And so, this is kind of a look at what I'm you know, referencing here. It's like you're raising 40 million, you need a 100 million contract over 3 years. You put down 30% prepayment. This is an illustrative example. You've got 10 10 million left to run the business, um and so you might have this kind of step function as you can see here. So, that's your contracted capacity and what you can actually in the demand is like, you know, uh below that. And so, now every time that you have to get into a new cluster, it's like tied to your equity round or whatever else it may be, but you might have to cap out and turn away customers. So, what are your options? You either turn away customers until you can get into your next, um you know, equity round so you can put down another 30% prepayment so you can get next little bit of capacity going, or do you use on demand to close the area under the gap and bridge the shortfall?

That's like an That's like an option now. Now you have to look into the economics. What does that look like? What's the difference between, you know, maybe instead what they do is they go to a lender to finance the prepayment component, but if the yield on the prepayment component that they're getting financed is less than, um them just buying it out on demand at that time if they're able to get into that amount of capacity, then does the economics make sense there? And so, here are some of the operating models that we've seen. It's like you can build and operate the full stack. Hardware, site, your team is going to arrange power, permit, the assembly, the management. You can also own servers in a colocation facility with more or less the assembly and operation handled by a partner. Um on the other end, an off-take contract buys defined capacity and the provider is going to own and operate the asset, and they might not give you complete root access to the bare metal or there might be kind of like levels to how much access you get depending on if it's tenant, multi-tenant, and etc. type of like configurations that can also affect them the pricing.

Um And so, some businesses are going to value the physical access, some businesses are going to value, um you know, confidential compute, uh you know, they might value certain topology changes or the ability to reconfigure the system. And so, if that is something for your business and they might actually want to have more and more control and build and operate the stack and the economics for them might make sense given the opportunity cost they would have to give up um if they were to just take rental off take contracts. So they might be willing to pay more there. And so here's how I ownership and rental is a very like crude graph. How ownership and rental can compare as utilization arises. And so what we're looking at is that usually you know an off take contract or these off take contracts are getting to the cost of actually almost building out and financing your own cluster. And it's not because I mean if you're spending a hundred million over a three year period then maybe if the economics make sense it would have made more sense to simply buy out the compute operator yourself outside of the time cost and then off the operational cost that might exist.

So there's a there's a equilibrium that exists that that has to be followed where you know the ownership cost outweigh the you know rental cost and the rental cost should sit below the ownership cost. But if you're utilizing at a such a rate and you know in such variable kind of increments it might be best to just own the stack out right. And so we're seeing this new dynamic that compute the compute procurement affects how much runway a growing company actually has. And so this also gives investors an interest in computer procurement. So we're kind of seeing this. Access and payment terms affect how much runway their port codes are going to have from a VC perspective. So helping a company buy a suitable capacity can improve the underlying investment and they're very connected. Right? So they're able to procure and they're able to actually find compute capacity for their port codes and and and fund that. And so that also brings another interesting aspect to the macro structure into the micro structure is that who helps place the capacity and how does this offer get compared and what risks these contracts can can transfer.

So we have a lot of different players around this GPU hour that are helping kind of shape this this market here. You've got your price discovery individuals. So, people with indexes, data, the benchmarks that we're all looking at right now and referencing, which would be used to, you know, in the future launch the futures based on the benchmark rates, you know, talked about in in the associated contracts per spec per pending approval. And then there's the physical matching venue auctions aggregators. So, anyone who can match capacity. We're looking at essentially decentralized marketplaces for on-demand compute. You're looking at liquid pools of compute that offer like liquid pool services and parted out themselves and have their own standardized contract. You're looking at bare metal resellers. You're looking at neo clouds themselves doing larger capacity sales. And then you're looking at individual brokers and individual operators here as well on the bare metal reselling. You have desk and brokers. They're looking at the bilateral quotes and execution. They might know a lot of people.

They might be able to move fast. They might be able to see quotes at very similar specs in one place and know that there's a buyer that is extremely inelastic to price right now. It needs the compute immediately and they can get the RFS in time so they can charge a decent spread on it. And then what they do is they're going to have the source negotiating and then they're going to be carrying and placing the risk. They're going to have to operate that cluster for the person and it has to be worth that spread. And you know, you have looked in financial markets, the exchanges that are coming that are based on the GPU hour, the futures and derivatives. People are going to be using it to take a look at how to maybe hedge out and look into more financial structures. So, hedge out risk. So, hedge out, you know, my balance sheet risk. Maybe get into more interesting agreements. We're looking we're seeing kind of agreements such as prepayment payment facilities are coming online. We're seeing people utilize futures and are interested in utilizing futures and other kind of um um options to be able to There's demand for this to be able to hedge out the risk that they currently have on the books. We have buy side individuals looking for flex contracts, flexible contracts like hey, is there ability that if I lock myself into a 3 to 5 year contract that I'll be able to you know, at least upgrade when a new chip that comes out or maybe I'll be able to roll it over or maybe let me sell it back to you.

And then there's a credit financing. So equipment lenders, leasers, specialist credit providers. And so the way we're seeing it right now is that every deal is bespoke. There's different spec sheets, there's formats. The bill of materials may not match the actual like delivery or there might be subject to change if a part is missing or part is out of stock and you know, there's like very long lead times that some of these OEM and ODM kind of providers have. And the same term can mean different things when you're negotiating very quickly. If someone's talking about memory, are we talking about you know, VRAM or we talking about the CPU memory, talking about additional storage here? Like what did you exactly mean there? It affects quite a bit what type of loads can be run on you know, the actual end users needs and also with pricing. And delivery dates. So there's a lot of back and forth before anyone can really compare and this is where the concept of like non-fungibility kind of comes into place and how to partition that out, what parts can be fungible, what parts hold the most of the weight such as the skew etc. And what parts are still like traded as if it was a product, e-commerce etc. where you know, it's only for that particular cluster that this price applies.

And so prices can update daily or faster. Quotes still a good starting point but it can get chopped rather than it is changes what's on offer. There's all these kind of latent factors, hidden factors as well why we're calling it a dark market. These are some public H100 references snapshotted today. I updated them. So take a look at the what the public so those are the references for if you wanted to go on their websites to see. This is just an example here of like what a public on demand or spot pricing looks like for an H100 today. You can see kind of various various prices here. And so, people might use this or they might use their own kind of internal what they've seen to actually like assume or use it to charge what they want to charge on their actual cluster specification, taking to account every other kind of factor that goes into it. And so, why are these new structures kind of needed and what are these microstructures interesting? It's mainly to solve for these mismatches, right? Bridging capacity, more flexible commitments, resale rights, and protection against price moves. Um you know, you're selling a fixed price service from an inference cloud point of view, you'd be selling a fixed price service while the GPU bill floats and that creates price exposure. And so, locking capacity for years can solve a part of that, but it'll create idle hour idle hour exposure elsewhere as well, unless they're always using it.

Reselling spare capacity only helps if someone actually can buy the hours. A financial hedge can address a reference price, but can't deliver the cluster right now in terms of cash settlement or guarantee a utilization. So, the useful structure has to start with a particular risk that the business has and needs to manage. >> And I'll I'll go pretty fast on this policy section because we're running out of time. Everything that a fan has said sits inside of a structure that is regulated and is affected by policy. In various ways. One way is export controls, one of the examples. In January, the US Commerce Department moved qualifying H200 class license reviews for China and Macau to case-by-case review with conditions. Furthermore, open weight models connect policy back to the US economics we started with. The US AI action plan explicitly encourages open source and open weight AI and recommends improving the financial market for compute so that startups and academics can access large-scale computing power.

>> So, I'm going to tie this out right here, but where's this opportunity lie? We have this growing middle market. We need balance sheets that can bridge buyer and supplier needs. There needs to be some level of standardization, a standard capacity contract, common specs, common terms, and definitions. And so, growing companies with needs and commitment do not fit the contracts the hyperscalers and frontier labs can sign. Those companies have real workloads. We want them to grow. They still face this financing problem, a timing problem, a specification problem. A supplier might want the long-term commitment for the build-out while the buyer wants to expand in smaller blocks as demand becomes clearer as it's so volatile, front of curve. The buyer needs a way to avoid paying for capacity can't use yet. And so, participants with strong balance sheets that are uncorrelated with the market and the ability to assess those risks with a strong credit profile can help bridge the gap between what buyers need and what suppliers offers. That requires more than quoting the actual GPU hour. It means understanding the delivery, workload, all of that. And so, there's very few kind of players in the market, and you know, a very well-suited player can be a firm that's uncorrelated with market, with strong balance sheet, strong credit rating, an incredible amounts of hardware talent that sits inside for high-performance computing.

And so, just making a CTA here, let's build this together. Thank you guys so much for attending our talk. >> [applause]