Written with Vibhu Sharma
A note before we start. What follows works through the innovation loops of the semiconductor world, sub-sector by sub-sector, with eight diagrams along the way. They are complicated, despite a fair amount of effort to make them otherwise. Anyone who wants to skip the loops and the slides can find the conclusions in two places: this opening section and the closing one about 4,000 words below, where the points that matter for money are stated plainly. That said, the loop diagrams are fun for those who relish complexity or those who want new reasons to be negative.
A core GenInnov belief is that innovation is the only true growth theme left in markets. A growing number of listed companies have realized that the key to better valuations in an AI world is to talk about new products, new services, and new business areas. Using the latest technology to shave a few points off costs is table stakes, and it wins nothing for long. So the number of companies attempting real innovation keeps climbing, first inside technology and now well outside it, at a pace that used to belong to a certain kind of venture company. One difference stands out. Most of what gets announced is a plan, and even in the best case, most plans need many years and a great deal of money before anyone can tell whether they worked.
The industry that follows innovation is changing with it. The days when a three-year forecast extended from current trends counted as analysis are ending, and innovation writing has moved from a handful of blogs into mainstream analyst reports. What has traveled less well is the habit of the corner. Most commentators pick a patch they can picture, and the patches that are easy to picture collect the attention: a new agent, a new chat interface, a new consumer gadget. The harder patches gather a following too, usually a specialized one that reads only itself. Two failure modes arise from this. The first is underrating how quickly a good idea gets copied when nothing protects it, as persistent agent products like Muse, Grok Bot, and Instinct are demonstrating publicly this week. The second is quieter. Innovation attempted is innovation achieved only some of the time. Physics does not yield to applause at a conference, and plenty of announced things will never ship, whatever the research or how charming the speaker. It is worth noting how often the word quantum turns up when somebody wants to signal that they are thinking big.
For anyone who enjoys a loop diagram—and the pessimists do love a loop diagram—the real difficulty lies elsewhere. Looking at one technology in isolation, and assuming that everything going right inside it leads to money, has become dangerous. Whether an innovation makes money now depends heavily on other innovations in the next field along, and often in fields far away. Some of those are prerequisites. Some are substitutes waiting to make the whole effort pointless. Some are customers whose own success decides whether anything gets paid for. Drawing arrows for money moving between a dozen companies takes an afternoon and proves very little. Drawing what has to be true in six other places before one technology is more complicated, but likely more important.
This piece stays inside one industry, semiconductor manufacturing and the infrastructure built directly on it. Similar loops run through equipment and materials, through models, through applications, and through biotech, and they deserve their own pieces. What follows starts with a master diagram of the whole spiral, then takes its seven buckets one at a time. Once again, anyone who tires of the technology is welcome to jump straight to the conclusion, about 4,000 words and eight diagrams below, where the points that matter for money are gathered.
The Spiral and Why It Turns
The master diagram is the last tidy thing in this piece. Its seven stations—compute, memory, packaging, connectivity, power and thermal, qualification, and capital—organize the problem. In practice, several move at once, and progress in one can change what is needed from the others.
Take memory. An accelerator waiting for data gains little from being able to calculate faster. Feeding it better can make more of its existing capability useful. But designing a system around that improvement brings other requirements into view: how the memory connects to the processor, how the assembly is manufactured, and how its heat is removed. The promised gain depends on several industries delivering together. Each has its own development schedule.
That dependence is what gives the spiral its shape. A constraint attracts effort and money. Relieving it makes a more ambitious system possible, which puts pressure elsewhere, and the money that arrived to relieve the first constraint has usually already committed itself before the second one appears. The next round begins at a higher level of bandwidth, density, power, and capital intensity, sometimes before the previous investment has earned its keep.
There is a less comfortable route around the picture too. Customers can respond to an expensive constraint by finding ways to need less of it. A memory producer may expand supply, a chip designer may change the architecture, and a model developer may reduce the workload's memory needs. All three can make technical progress. Their business plans may be less compatible.
This is where the arrows become useful for investing. A supplier's opportunity depends on how long its contribution remains necessary, how readily customers can substitute for it, and whether the rest of the system will be ready in time. Scarcity can create pricing power. It also gives customers a reason to finance an escape.
Read the diagrams with those competing responses in mind. The maturity labels matter because an available workaround can shape purchases while a more elegant solution is still being qualified. A place on the same page does not mean a place in the same purchasing cycle.
Compute Has One Incumbent and Many Forks
The compute diagram is crowded because several different arguments are taking place at once. Some challengers want to build a better general-purpose accelerator. Others want a chip tailored to a narrower workload. Still others aim to get more useful work from the hardware already installed. Success in one route changes the market available to the others.
Start with what the challengers are challenging. The cadence is now annual, and the product being sold has grown from a chip to a rack of dozens of them, tied together by a proprietary fabric. A competitor therefore has to match a system—its memory, interconnect, and software—which is a far larger undertaking than matching a part. That explains three things on the diagram: why the same few design houses sit behind most custom programs, why a national stack built under export controls counts as a separate branch, and why some challengers end up licensed into the incumbent instead of beating it. Absorption pays differently than winning or failing.
Consider the pull toward custom silicon. At sufficient scale, a cloud operator has a reason to design around its own workloads and seek better economics. But the finished design still needs manufacturing capacity, memory, packaging, chip connections, and software to make the system useful. The commercial question extends well beyond whether the chip works. It includes how much of the hoped-for saving survives those requirements, and whether the workload remains suitable when the system arrives.
Specialization makes that timing question sharper. A design optimized for one pattern of computation may perform very well, and its investment case depends on that pattern lasting long enough to recover the development cost. A chip that wins by hardwiring a model architecture is exposed to any change in model design, and the shift toward sparse models, where only a fraction of the parameters is active, is already underway.
The efficiency routes further complicate the calculation. Lower precision, sparse models, and separating the stages of inference seek gains by changing how the work is done. Their implications differ across workloads, and they force the same forecasting question: how much hardware will be needed for a given amount of useful output? Lower costs may then encourage greater usage, so a gain in efficiency alone does not settle the demand forecast.
Several competing designs may draw on the same constrained suppliers. Others may change the type or amount of memory and connectivity required. Counting announced accelerators tells us little about how those demands will add up.
That takes us to memory, where efforts to supply more and efforts to need less meet in the same diagram.
Memory Is Where the Scarcity Sits
Memory used to improve in one direction. A new process packed more bits into a die, and everything else followed. High-bandwidth memory now advances along five vectors at once. Faster raises the signaling rate on each connection. Wider doubles the interface to 2,048 connections, buying bandwidth at the cost of routing density. Taller moves stacks from eight dies to twelve and sixteen, adding capacity with no additional footprint and shifting the problem to thinning, bonding, and assembly yield. Smarter builds the base die at the bottom of the stack on a logic process, so it can absorb control, test, and repair functions. Repartitioned moves the boundary between accelerator and memory subsystem itself, a system architecture decision wearing a memory costume, and the one our earlier piece, Memory by Design, followed furthest.
The five vectors overlap. One generation ships in volume while the next ramps, samples of the one after that reach customers, and more ambitious concepts appear at conferences. They compete for engineering effort and capital on very different schedules. A customer choosing a system today must work with the memory that can be delivered and qualified in time.
Scarcity adds a second set of responses, and part of its cause is physical. A stacked part consumes roughly three times the wafer area per bit of ordinary DRAM, so every order for it squeezes everything else made in the same fabs. Memory producers work to supply more bandwidth and capacity. Their customers work to get by with less of the expensive kind. The diagram follows both, because success on either side changes the opportunity on the other.
Some responses stay within HBM, using shorter stacks and spreading the workload across more accelerators. Others change where data sits. Placement software decides what must remain close to the processor and what can move into cheaper memory or storage. Model designers compress the key-value cache and reduce the memory needed for weights. Each route has limits: a saving in scarce memory may require more data movement, more hardware elsewhere, or a compromise in performance.
This is where the loop becomes commercially awkward. A supplier expanding capacity must judge how much demand will remain by the time that capacity arrives. A customer developing a workaround must judge whether the saving will still justify the extra complexity. Greater availability and lower prices can weaken the case for substitution. Persistent scarcity can help a workaround become an established design choice.
Taller stacks put pressure on bonding, cooling, and assembly yield. Moving data farther from the accelerator increases the burden on connectivity and placement software. Memory innovation therefore changes both how memory is built and how the rest of the system uses it. Packaging sits at the intersection of several of those choices.
One exclusion is worth naming. The storage side of the hierarchy appears here only where it reaches back toward the processor, in flash placed behind a memory interface and in context offloaded to SSDs. Everything below that line has been left out: NAND scaling past four hundred layers; cells and control circuits built on separate wafers and bonded together; controller design; and the interfaces that move data in and out. Those are loops in their own right, with their own substitutions and their own suppliers, large enough to need a diagram of their own. Treat this picture as the part of the hierarchy that competes directly for the accelerator's attention.
Packaging Became the Computer
Packaging used to be treated as the last step before a chip left the factory. It now decides how much computer fits in one place, giving it a scaling roadmap as ambitious as anything in logic. It also brings more of the system's value into a single assembly, where a manufacturing problem can become costly.
The lateral race is about area. Larger packages provide more compute and memory, provided the connections between them can be manufactured reliably. Current parts already span several times the area of a single lithography field, and the published roadmaps go beyond nine, with a dozen or more memory stacks, before the end of the decade. Those are supplier roadmaps, and the whole industry is planning against them. Alongside sit alternatives with different economics: local silicon bridges, larger manufacturing panels, glass substrates and additional capacity at outsourced assembly houses. Each offers a route to a larger area or lower cost, and each must establish that it can deliver enough good packages to make the savings real.
The vertical race is about pitch. Bringing dies closer together allows denser connections and makes surface quality and assembly precision more demanding. Copper-to-copper hybrid bonding opens possibilities beyond conventional micro-bumps. A fine-pitch research demonstration, a production tool, and a process qualified for a customer's product are separate achievements. Progress in one does not establish the other two.
Nor does the most advanced method have the timetable to itself. If a change in permitted package height gives an existing bonding process more room, customers may be able to reach their next capacity target without adopting its successor. A relatively modest change elsewhere in the system can extend the commercial life of an older technology. The new process then has to compete against an incumbent that has bought itself more time.
What moves into the package makes the choices harder. Memory stacks bring height and thermal requirements. Optical engines bring connections that must be assembled, tested and kept reliable. Changes in power delivery alter the electrical and physical design, and more ambitious cooling proposals bring the thermal solution into the assembly itself. Each changes what the package must accommodate and how it can fail.
More integration promises better system performance, encouraging larger and more complex assemblies, whose manufacturing demands raise the value of bonding, inspection, testing, and thermal control. By the time a package is assembled, it carries a great deal of known-good silicon, so a late failure scraps everything bonded with it. If cost or yield disappoints, customers may change the design, retain an older process, or place some functions outside the package. Those choices reshape demand for the capacity and equipment being built to serve them.
The durable advantage may therefore lie with whoever can co-design the interposer, bonding, thermal path, and inspection as a single yield-learning system. The evidence is how reliably an ambitious design becomes a shippable product, and at what cost.
Connectivity Has Many Paths to Light
Optics has been the future of short-reach links for twenty years. It is arriving along several paths at once, which makes the investment question harder than simply predicting more light.
The driver is the amount of time an expensive system spends waiting. More compute within a node increases the demand for communication between processors. Faster chips deliver less than their promised gain if data movement and synchronization cannot keep up. Packaging brings more compute together; connectivity determines how effectively it works together.
As electrical lane speeds rise, loss and the power needed to recover signals make longer copper paths harder to sustain. At around 200 gigabits per second per lane, that cost becomes hard to justify over any distance, and the date at which each link crosses that threshold sets the timetable for everything else on the diagram. Optics moves closer to the chip through competing designs with different compromises. Conventional pluggables preserve easy replacement. Linear-drive optics remove processing from the module and place more demand on the electrical channel. Near-package optics shorten that channel while retaining some separation. Co-packaged optics bring the optical engine into the package, making optical assembly and repair part of a much more expensive system. These routes can coexist across different uses while competing for the same design decisions, and putting their bandwidth numbers side by side can make the contest look more settled than it is.
The manufacturing dependencies multiply as optics moves inward. Silicon photonics, drivers, fiber attachment, and thermal control must work together, with different suppliers responsible for different parts of the result. Lasers deserve separate attention. They are built on indium phosphide, their capacity is tight, and they are the component most likely to limit how quickly more integrated designs can scale. One reason a socketed engine with an external laser source exists at all is to keep that constraint outside the expensive package.
Standards add a second contest over how processors talk to each other inside a rack. This is the compute argument reaching its next station. A proprietary fabric ships in volume today. Two open alternatives are working to displace it. The question is whether an open standard can arrive in time to matter while the incumbent is already selling complete systems. Optical circuit switching sits alongside all of this, reducing packet-switching requirements where traffic patterns allow.
Faster links allow larger compute domains, whose communication demands encourage another increase in link speed. Moving optics closer reduces the electrical burden while raising the importance of alignment, attachment yield, laser reliability, testing, and repair. Progress transfers some of the difficulty from moving signals to manufacturing and maintaining the system.
Copper keeps changing the timetable through active cables and shorter electrical paths. Every extension of its useful life gives customers a reason to postpone an optical transition, and a workable intermediate optical design can delay a more integrated successor in the same way. The investor therefore needs a view on which links change first, which components those designs require, and how long they remain useful before the architecture changes again.
Power runs through every one of those choices.
Power and Thermal Set the Schedule
Rack power is climbing, and the arithmetic is unforgiving. A megawatt at 54 volts draws about 18,500 amps; at 800 volts, about 1,250. For the same conductor resistance, losses rise with the square of current. Higher distribution voltage reduces the burden on copper, while moving major conversion equipment outside the compute rack frees space inside it.
The diagram separates two changes that should not share a maturity label. Sidecar power racks relocate conversion equipment and are close to deployment. Solid-state transformers attempt a deeper redesign, converting medium-voltage AC to the DC supply the system needs. This requires switching devices, magnetics, protection, controls, and cooling to advance together. Progress in power semiconductors helps, but an available device is still several qualifications away from a usable power system.
The electricity then becomes a cooling problem. Heat must cross the chip, the cooling interface, and the facility before it reaches the outside world. Direct liquid cooling is now the standard for dense racks, specified with warm water so mechanical chilling can be reduced where conditions allow. What follows is genuinely open, with two-phase cooling, immersion, and coolant channels inside the silicon, all shown on the diagram, and none of them settled. Whichever wins will need connectors, distribution equipment, and service procedures that customers trust, which is a slower business than improving the thermal physics.
The underlying difficulty is synchronizing historically different clocks. Accelerators, cooling, converters, and rack interfaces increasingly have to meet the same deployment date. Utility connections, protection standards, and lifetime qualification have not kept pace with semiconductor cycle times. Their decisions nevertheless have to be made against the compute roadmap.
The largest loop on this diagram runs through the grid. Denser computing raises power demand, while delays in grid access encourage builders to seek their own generation or secure existing supply. Those alternatives bring their own equipment queues, fuel requirements, and approvals. Batteries can bridge interruptions and shift demand, and they still need an energy source. Flexible workloads offer another bargain: accept some curtailment in exchange for an earlier connection.
Each response changes the next investment decision. Better cooling makes a denser rack feasible, and its power requirement may change the site or the opening date. On-site generation can shorten one wait while adding another. What matters is how soon the whole installation can operate reliably, and how much useful compute it can deliver once it does.
Qualification Is the Real Gate
Capacity counts only once it is qualified. A new packaging line or memory supplier adds little usable supply for a customer until its parts pass that customer's requirements. As packages combine more dies, the distance between installed capacity and dependable output can grow. A factory opening is easier to announce than a reliable yield.
The expense accumulates during assembly, and the order of operations makes it worse. Memory stacks go in early, so everything bonded afterward sits on top of a part that was screened before the expensive work began. A defective die or bond discovered late can put the entire package's value at risk. Known-good-die screening catches problems before more silicon is committed, and testing between assembly stages checks what the next step might otherwise bury. Burn-in stresses parts to expose early failures, and some of it is moving from finished systems to the wafer, shifting the work and spending to a different point in the chain. System-level test runs parts under realistic workloads, because passing an electrical check does not establish that a processor will behave reliably in a server. Testing high-power parts then requires substantial power delivery and cooling.
Inspection has to reach places that assembly hides. X-ray and acoustic methods look inside stacks for voids and delamination. Finer bonded interfaces demand tighter measurement and defect detection, and automated classification helps sort the resulting flood of images. The difficulty is finding consequential defects quickly enough to keep production moving. Longer tests improve confidence while reducing the output of a given set of machines.
Qualification also allocates market share. A memory supplier that clears a customer's requirements earlier can secure orders while a rival's installed capacity remains unavailable to that program. New substrates, optical assemblies and liquid-cooling systems face their own reliability tests. A second source offers little bargaining power until the customer can actually use it.
For equipment suppliers, spending arrives at different times, and the split is now visible. Electrical test is collecting orders today because it scales with production that is already running. Demand for each inspection tool depends on which bonding processes reach volume and which measurements customers require in production. Some inspection steps become more intensive, while others must become faster or more selective to remain economical. More complexity creates opportunities, and it does not make every proposed test a permanent addition to the line.
The feedback continues after shipment. Large fleets reveal silent data errors that factory tests missed. Those findings inform tighter screens and revised designs, which improve reliability while adding test time or reducing the share of parts that passes. The supply number changes without a single factory being added or removed.
The scarce resource is therefore reliable output at an acceptable cost. Installed capacity is only the starting figure.
Capital Follows Scarcity
Every bottleneck in this spiral draws money. Scarcity supports high returns; those returns encourage spending, and new capacity changes the conditions that justified the investment. The factory takes years to build. Its selling price can change a good deal faster.
The financing now reaches well beyond cloud cash flows. Bonds, sovereign capital, private equity, and private credit spread the cost across more balance sheets. Accelerators become collateral. Multi-year leases and take-or-pay contracts turn expected demand into commitments that lenders can use to lend. Suppliers invest in customers, and developers contract power before their buildings exist. These arrangements help projects proceed earlier, while tying repayment to decisions elsewhere in the system. A long contract allocates risk; it does not eliminate it.
The money reaches memory fabs, leading-edge foundries, packaging lines and power equipment. Each expands on a different schedule. A completed building may wait for electricity; installed equipment may wait for qualification. Meanwhile, alternatives developed during the shortage can change what customers want by the time the new capacity arrives. Capital can relieve a constraint and still earn a disappointing return.
The first difficult question is how long the assets earn. An accelerator's physical life, its accounting depreciation, and its competitive earning life are three different things. Older hardware can remain useful while commanding a lower rental price or consuming more power per unit of work than its successor. Whether it earns adequately for three years or six matters to its owner, its lender, and anyone relying on its collateral value. Extending a depreciation schedule changes reported earnings. It does not secure another customer.
The second question is circularity. Supplier investment and vendor financing can help customers adopt useful technology. They can also make orders look more independent of continued funding than they are. Calling the arrangement circular settles neither point. The work is to follow the cash: who ultimately pays for the service, whether that revenue covers delivery and financing costs, and which participant must raise more money if it does not. Contracted revenue is valuable, and its quality depends on the customer's ability to honor the contract. In several cases, that ability depends on the customer raising fresh capital from investors who read the same contract as evidence of demand.
The hopeful outcome is that cheaper capability brings enough additional paid usage to support the investment. The harder outcome is that usage grows while prices fall faster than costs, leaving impressive activity and inadequate returns. Both can accompany rapid technical progress. The spiral can keep turning even when some of the capital that financed it does not come back.
The Conclusion, For Those Who Skipped
Eight diagrams later, the honest summary is that the picture is a mess, and the mess is the finding. By the time you finish reading, a few more daring plans will have been announced and congratulated as achievements. Asking about the probability of delivery, at the proposed cost and on the promised date, can spoil the occasion. Mentioning a competing announcement that makes the whole undertaking less useful is positively rude.
The vocabulary makes it easier. A roadmap becomes a schedule, a sample becomes a shipment, an architecture becomes a product, and a demonstration becomes a capability, all without anyone saying anything untrue. Physics has not agreed to any of it. Physics does not attend conferences, has never been moved by a keynote, and remains unimpressed by the word quantum, which turns up most reliably when somebody wants credit for thinking big.
Several of these technologies can succeed while undermining one another's business plans. Scarcity finances both more supply and ways to need less of it. Today's indispensable supplier may be tomorrow's expense to engineer out. Success buys resources for the next contest. It buys no exemption from it.
So we want evidence before commitment, and room to change our minds afterward. Qualification results, repeat orders, yields, and cash collection carry more weight than a launch date. Waiting costs some upside. Paying a valuation that assumes every unfinished step will work can cost considerably more.
Keeping the exits open matters at least as much. A sound investment can become unsound because somebody else succeeded, which is a strange way to lose money and a common one. So we keep checking what customers require, what competitors can replace, and whether the original economics survive. Loyalty to a thesis is expensive in our Super-Moore era.
That is the particular attraction of public markets. They let us stay current with an evolving technology landscape and move our exposure as the evidence changes. No technology or product endeavor stays certain of success for long, and the ones that look certain are usually the ones being priced as though the next six dependencies are already solved.
And what a future it is attempting. This is an extraordinary era of innovation, increasingly built through scientific and engineering undertakings far beyond the garage and the modest seed check. New materials, intricate manufacturing, optical links and power systems require vast capital and coordination across disciplines. The ambition deserves celebration. Much of what these teams are attempting is astonishing.
We can applaud the attempts while recognizing that a meaningful number of announcements are turning more bombastic and impractical.
