Itamar Friedman
Itamar Friedman

@itamar_mar

8 Tweets 40 reads Jun 19, 2023
pick your AI programmer friend πŸ€–:
'gpt-engineer' - @antonosika
'smol-dev' - @swyx
'AutoGPT' - @SigGravitas
'Metamon' - @yoheinakajima
they commonly "work" this way:
β–Έ You give a first set of instructions
β–Έ AI asks clarifying questions, generates spec, writes code, πŸ”
πŸ‘‡
but do they actually work?
Yes. No. Maybe so.
how can we know?
we can define a set of challenges, use them as a benchmark, and even practice challenge-driven-development
some of the makers of these AI programming friends are doing this
e.g. @Auto_GPT:
gpt-engineer made a couple of simple programming tasks and uses them as the benchmark
attached is a snapshot (2023-June-19)
e.g.
gpt-engineer can quickly develop a currency_converter for you,
but can't properly code a pomodoro_timer (yet!)
@yoheinakajima seems to be on a roll, and made several agent-like projects
most famous is the @babyAGI_
and just recently, Metamon
I didn't see how we can try out with Metamon -- @yoheinakajima??
If you read threads on twitter/discord/github,
you see developers trying these projects and generally sharing this feedback:
β–Έ The potential is very exciting
β–Έ Sometimes work for [trivial] projects, and saves you a lot of time (at least creates a lot of boilerplate for you)
1/
2/
β–Έ For most projects you still need to know what you are doing
β–Έ Requires a continuous feedback loop between the developer and AI
β–Έ Mostly suited to start new projects --> but I assume that new projects will aim to address this pain and work better with an existing codebase
Read smol-dev instructions or watch the demo video (by @swyx) to get a pretty realistic view of what can be achieved these days

Loading suggestions...