Journal · Simulation
Four walls we hit, and what each one would take
Everything we could not finish, sorted by the kind of thing that stopped us. Only one of the four is about money.
At the end of a piece of work the interesting list is not what you did. It is what stopped you, because that list tells you what to buy, what to build and what to go and learn.
Ours has four entries and they are four genuinely different kinds of wall. One is an artefact we cannot obtain. One is a method that does not exist in our code. One is a budget. One is a property of the problem that no budget fixes. They are worth separating, because the instinct is to treat every blocker as the third kind and throw money at it.
Wall 1: a binary nobody will sell us
Our pipeline reaches 5 of its 6 stages. The sixth is not missing because it is hard. It is missing because the reference output we would check it against was produced with a tool we have found no copy of, for any platform, free or paid.
The stage itself runs here using an open substitute. What it cannot do is prove it ran correctly, because the only thing to compare against was made with the tool we do not have. That is a ceiling on verification, not on capability, and the distinction matters: we could use this stage today and simply not be able to tell you whether it agrees with the authors.
What it would take: an email that gets answered. We have not yet sent it; it is the next item on this list. Until it arrives, 5 of 6 is the honest number, and we would rather publish 5 of 6 than quietly count a stage we cannot check.
Wall 2: a method we do not have
This is the scientifically interesting one.
Our sampling works by running copies of a molecule at a ladder of temperatures and swapping neighbours. For a small protein it works well. On a 213-bead protein it made zero round trips in 24,096 exchanges, with swap acceptance that looked completely healthy the whole time.
The reason is not a tuning problem. Larger proteins fold through a first-order transition, which has latent heat: at the transition temperature the molecule must absorb a large chunk of energy before it can cross, and the two states are separated by a gap in energy rather than by a gentle hill. Putting more rungs on the ladder does not help, because the rungs are not the obstacle. The gap is.
So every protein above roughly 200 beads is gated on a sampling method we have not implemented. The candidates are well known, multicanonical sampling, Wang-Landau, or well-tempered metadynamics, and all of them work by flattening the energy landscape so the gap stops being a gap.
What it would take: implementing one of them and validating it against a case where the answer is already known. This is the only wall on the list we would be pleased to hit, because it is the one where the work produces something we did not previously know how to do.
Wall 3: a budget, and a small one
Two pieces of work are blocked purely on rented compute:
| Work | Cost | On this machine | Rented |
|---|---|---|---|
| Ten proteins, concurrently | about $79 of spot time | not practical | 150 cores genuinely busy |
| Four further trading cards | about $252 | 33.6 days | 4.8 days |
Note what the second row says. The constraint is wall clock, not money. Three hundred dollars is not what stands between us and that result; a month of calendar time is, and the rental converts the month into five days. That is the whole argument for renting compute, and it is an argument about time rather than about capability.
What it would take: a card and a decision. This is the least interesting wall on the list, which is exactly why it is worth separating from the others. It is the one people reach for first.
Wall 4: a ceiling that money does not move
The fourth wall is the one that surprised us, and it is the reason the batch of ten above is ten proteins rather than one big one.
The obvious way to show that a method scales is to run the largest protein you can. We costed it: 25,637 core-hours. Rent a thousand cores and it still takes 71 days, because a single replica-exchange run saturates at roughly one core per temperature window, about fifteen of them. Buy a thousand cores for that job and 985 of them sit idle.
Ten proteins at once use 150 cores genuinely concurrently. Same money, a hundred times the parallel efficiency, and it demonstrates the thing that actually matters for anyone deciding whether to run this on real work.
What it would take: nothing you can buy. The ceiling is in the shape of the algorithm, and the only ways past it are a different algorithm, which is wall 2, or a different question. We chose a different question, and we think that was the right call rather than a concession.
The pattern in the four
Lay them out and they sort by what the fix costs:
| Wall | Kind | Fixed by |
|---|---|---|
| Missing binary | artefact | somebody answering an email |
| First-order transition | method | implementing and validating a known technique |
| Ten proteins, four cards | budget | about $79 and about $252 |
| Single large protein | structural | nothing purchasable |
Only one of the four is money, and it is the cheapest item on the list. The expensive ones are an artefact we cannot obtain and a ceiling we cannot buy past. That ratio is, in our experience, the normal one, and it is the opposite of how compute-bound research is usually described.
What this does not establish
It does not establish that these are the only four. They are the blockers we have recorded. The list is written by hand from the records behind the progress bars on the main study, so it can lag that page; the two costs above are read from it when this page is built. Walls we have not hit yet are not on it.
It does not establish that wall 2 is solvable by us. The three candidate methods are published and well understood, which is not the same as saying our implementation would be correct. The validation is the work, not the implementation.
It does not establish that the $79 and $252 figures are what you would pay. They are spot prices for interruptible capacity, computed from our own measured core-hours on one machine, and spot prices move.
Why publish a list of failures to finish
Because the alternative is a results page that silently implies the method goes further than it does. Every one of these four walls is a thing a reader might hit on their own work, and three of them we can describe precisely enough that they would recognise it sooner than we did.
The full study, with every number traced to the file that produced it, is the protein folding write-up. What it took to get the published code running is in running somebody else's research code, and the six corrections we made along the way are in six times we were wrong.