Bolt First Exp

What can be said about bolt.new after 20 hours of use and creating 2 applications:

  • I managed to make a simple project for displaying data from Ansible inventory files as a table. Right from the first run, it created a working project that showed everything needed and looked great. It took 2-3 hours to decide what I wanted and implement it. Doing this myself would have taken 1-2 days.
  • A medium-sized project for a universal ChatOps bot based on an LLM could not be completed, even though I have a detailed specification and know how it should be structured. Bolt constantly gravitated towards web development, even though it was secondary. it kept trying to write in JS instead of TS. Each iteration took a lot of time and was expensive. In the end, I got tired of struggling with it.
  • It is not yet suitable even for medium-sized projects, as there are obvious difficulties with the context size.
  • Bolt is practically not integrated with their own online IDE StackBlitz, to the point where code can be lost.
  • The infinite chat interface, in my opinion, is not the best idea, as the context length keeps growing.
  • Sometimes you have to fight the model. For example, it constantly tried to use GPT-4 instead of GPT-4o-mini, even though I explicitly specified it. In general, the model is very stubborn, and it’s hard to change its mind.
  • It’s very focused specifically on code generation. In response to questions about how to do something, it suggests generating code.
  • The generation speed is typical for modern models. You ask for a change and go do something else. Sitting and watching the generation for 1, 2, or 3 minutes is not very interesting.
  • After each modification, the application tries to start, but it’s unclear exactly which command is used. Eventually, everything broke, and I had to run the build manually.
  • It sees the logs but often fails to identify errors in them. You have to give it hints.
  • Changes made by the agent can be quickly rolled back. Works great!
  • It does not generate tests unless asked.
  • It lacks error highlighting directly in the code. But again, this relates to IDE integration.
  • It works much better with TypeScript than with JavaScript. This makes sense, as it requires less context.
  • It starts to get lost if there are many edits. At one point, the static analyzer found 20 errors at once, and it took about 10 generations to fix them.
  • A basic paid account is enough for the whole day if the project is small.

This is the next level after Copilot and even Zed. It’s the closest thing to how IDEs should be structured in the future — a full-fledged assistant that writes code according to a task. But it still has “teething problems.” I would like something faster, more integrated into the IDE, and more interactive. And preferably locally in my IDE, not in the cloud. It’s actually surprising that JetBrains hasn’t done anything like this yet, instead focusing on trivialities.