Haskell šŸ’œ Vibes / Jappie

I feel you’re underestimating the ā€œlevel of detailā€ you can work at with these things.
I asked it to make it ā€œto store the hardcoded level format as toml files, and add round tripping read write tests with arbitrary levelsā€

It halucinated this entire Level module and associated types, it looks pretty reasonable to me. (tbf it tries to copy my style).
Even if a TilePlacement for example is redundant compared to an axial, it’s not the worst idea to have custom types for your stored format so you can have stability accross level formats.

It spend like 20 minutes on it, worked first try, I just watched movies instead. :man_shrugging:

oh the round trip tests were a later prompt here: Add property test for Level TOML roundtripping Ā· jappeace/plunder@9e29d98 Ā· GitHub

Oh I don’t underestimate. But that commit is just munging, and a lot of the code reads like unrolled abstraction.

For this sort of thing maybe it’s fine? idk. you just want a TOML version of your levels that works ig. Doesn’t feel like there’s anything especially transformative or even Haskell specific in this specific code. Definitely falls within my estimations of what Claude can do.

Although for such a thing, I could also probably just type it myself in like 30 minutes? and the code will probably be a bit nicer to work with cuz I’m working with it. And that process does sometimes reveal multiplicative opportunities to write high leverage code. (For instance, I would have less of an appetite for boilerplate than Claude, so I’d probably whip up something a bit more type directed and declarative for the serialization)

hah this is funny to me because i kind of already always worked like this anyways? way before LLMs existed. except in reverse i guess?

i’d plan my haskell, then go play 20 minutes of melee or do a chore or something else physical while the haskell part of my brain simmered in my subconscious. For harder problems, i’d let it idle for days. Then I come back and the code comes out in a trance without me being super present either (since the hard thinking is usually done and the rest is just domino’s falling.)

4 Likes

Ah, hammock driven development. I’ve never really been able to do that (perhaps I’m not giving myself enough time).

6 Likes

Gotta make sure my Claude bill is in the peloton to avoid suspicion

I identify with this so much :melting_face: everyone is looking at Cursor leaderboards daily to see where they stand…

2 Likes

You’re right, but that hill has been lost long ago. It is assumed that LLMs have agency and the language we use reflects it—you yourself said ā€œit liesā€ in the paragraph above, knowing full well the system has no notion of truthfulness in it. Similar thing happened with cryptocurrencies, where everyone assumed you can ā€œpayā€ for things with them, while the process internally has no way to compel the counterparty to anything (your ā€œpaymentā€ is effectively a gift).

The Internet of today is driven by algorithms, and outrage is a great way to maintain engagement. Expect more ludicrous things just around the corner, perpetually.


It’s not important to your employers because they have business goals and no knowledge of software design. The ā€œhowā€ is important to you because that’s how you get software that’s concise and easy to maintain, making your job easier in the future.

* Yes, there are jobs out there that just need garbage software for garbage reasons, and, no, I don’t want to work there.

So you trust the LLM to design the tests then?

You don’t have to religiously hammer error fixes into the codebase, you could instead reconsider the overall design to reduce code overlap. Breaking your code in 30+ places on a simple change does not sound normal.


Choose one. (politics determines who gets to spend whose money)

3 Likes

That’s like saying you can still navigate without Google maps.

I don’t think this is going to be true in the long run. I’ve already linked it in my blog post, but I’ll do it here again:

The theory is that there are two types of thinking: 1. fast, dealing with high throughput and 2. slow and methodical. They’re both useful, but especially for learning and difficult problems, we need the slow and methodical thinking. AI does not facilitate this type of thinking though.

And that’s the crux: it’s not the same as the revolution of navigation apps, where we simply lost a skill and have tech replace it forever. This time it’s about our ability to think. And it’s not just junior developers and kids in school. They just have a worse start. The brainrot is also going to affect senior engineers today, at least that’s my opinion.

12 Likes

As someone who only began to write code in any sort of real anger ~4.5 years ago (aged 24, now 28, no computer science background outside of GCSE ICT at 14 years old and messing about with twitch and PC gaming since then, hell I haven’t even done Maths post-16). I feel in a very weird position compared to the industry.

I’ll be very honest when I say this: I hardlucked out getting my $DAYJOB about three years and a half years ago. I staked everything on them, they took a chance on me and to be honest, it’s been the best thing that ever happened to me, but every day I have to overcome a ridiculous amount of impostor syndrome and continue to learn and execute things at my own pace. My lovely colleagues, some of whom have been living and breathing this stuff since they were actual children, some with my entire lifetime’s worth of experience in this industry - and it shows - are immensely helpful; also immensely daunting. I started off with just Haskell; work made me learn C, then Python and finally C++ (ew) and still to this day, Haskell is my favourite.

The technical interviews I faced when everyone was hiring about three/four years ago, I’m fairly confident that I could singlehandedly smash today in an afternoon but that was three years ago and back then, I really did not know what I was doing and employers seemed to ask for a lot less. In fairness, one thing that will always stick with me was one interview where I asked if I could use HLS/ghc" and the interviewer replied ā€œDon’t worry about it, I will be your compilerā€ - it’s only in the past year where I have become confident working without HLS to write Haskell code (for smaller snippets and scripts).

This past 1.5 years has been a constant battle of seeing people allegedly ā€œbuildā€ all these incredible complex things with ā€œAIā€ and talking about ā€œif you’re not doing this, you’re getting left behindā€ vs me struggling to even know what the basic elements of what they made even consist of vs my actual $DAYJOB where hilariously enough, I can’t even use many of these tools in this manner for various reasons.

In that same timeframe, my C/Haskell/overall computer knowledge has dramatically improved: partially through self-study, my efforts with the exam board, work etc. I would like to do a computer science Masters one day just for the love of learning.

For example, right now, I’m building a rendering engine in Haskell from scratch with SDL because I want to understand the core internals of it and solidify my linear algebra skills properly. In brief interactions with Gemini, I’m under no illusions that if I left Gemini to build it, it would have done it just fine but for me, I want to understand the rendering pipeline so that I can always reimplement and modify it to my own desires and I can only feel I can do that if I know what it takes in the first place and can reliably replicate it.

My most recent endeavour, I did a similar thing. I wanted to finally understand type families and GHC.Generics and so I learned it. Now I reckon I can wield both at a reasonably competent level since I can independently replicate the solutions outlined in that thread from memory and intuition.

This is where ā€œAIā€ has me in a real pickle. How can I leverage it in a way that keeps me effective but also does not put in a bind getting me to appear to ā€œsolveā€ problems I do not fundamentally understand? Right now, I see it as a 20x multiplier to whatever my base level and in order for me to maximise whatever benefit it can give me, I need to increase my base competency first and vigourously.

I have had advice in all directions from many people with varying stakes in me.

I volunteer at a local school teaching 11-14 year olds various ā€œcomputeryā€ aspects with some colleagues, I even shoehorn in some ā€œThinking With Typesā€ stuff for the love of the game. For every kid that breaks my heart and just tries to get me to tell them solution that’s already provided for them on the board - or worse - ask one of the LLMs they have available, there is another who will take the model solution, poke and prod it and test its limits and learn with it. For the first group, I wonder what can I do? I only have an hour with them a week, and that’s if I’m not getting slammed with stuff at $DAYJOB.

It was funny seeing those memes about jobs asking for X years of experience for things that existed for less than X years, yet nearly 5 years in, I kind of get it. At the same time, the only reason I’m this good is because I’ve been able to gain that experience in the first place by dedicating that much time and energy to solving the problems I’ve been given. I feel the rush to create AI-first developers can jeopardize that if we lose all the juniors who are comfortable to work around problems in analogue manner.

I don’t know what the future holds for me to be quite honest: all I know is that right now, for me personally, I’m lucky to be in a situation where it’s not expected or mandated for me to use AI heavily in my work. It doesn’t mesh well with my personality or risk tolerance. Still, I appreciate those times where it does break the ice for me on something that I’m fundamentally undestanding poorly or am encountering significant inertia to otherwise overcome; at the same time, I also yearn for better access to freely available materials that could the same - something I hope to provide myself one day.

18 Likes

This is going to become a real problem for open source projects. My understanding is that the LLM generated code does not have a copyright and determining whether someone has made enough of a modification to be ā€œtransformativeā€ from the perspective of copyright is going to be difficult without lawyers. I genuinely don’t know how to resolve this without wishful thinking and pinky promises.

2 Likes

I share many of your concerns but I’m also an active user and I don’t think conflating the economic, social, cognitive and technological dimensions helps with understanding. You can run an LLM on your desktop workstation, in particular one trained with fully open and/or synthetic data.
As for the shifting of labor bargaining power to capital, that didn’t start in 2023 with ChatGPT as I’m sure you’re aware.

2 Likes

Not one that comes even close to being useful for anything more than auto-completing individual lines or at most small boilerplate snippets, in my experience. I haven’t even restricted myself to ethically trained models, just free local ones.

1 Like

I agree, but generation quality is orthogonal to the other factors though. If you want all of quality, speed and self-hosting you need hundreds of GB of high-bandwitdh memory with dedicated interconnect and a frontier open model like K2.

The technology, per se, exists and it’s ~freely available for experimentation. If you are not driven by curiosity to experiment with it, it’s likely the case that some economic factor is pushing you to use it (your employer or their competition); in that case you are pressured to get the best tools money can buy, including (as the most common path of least resistance), renting tokens from a hyperscaler.

1 Like

I see a lot of polarization about this in programming forums (including this one), so I find it necessary to add some nuance.

There’s a whole spectrum of activities between ā€œhands-off vibecodingā€ and toiling on every keypress.

As @ApothecaLabs shared, you can use LLMs to refactor sections of your code. Or you could use it as a high-level design partner when you’re out of ideas for a gnarly integration. You can generate a design document, study and modify that, generate code from it, review it and repeat until satisfied. There’s converging evidence that LLMs can be a useful accessibility tool, or a ā€œprismā€ for viewing ideas from different angles before committing to a single one. To say that all of this is a net negative for the craft is disingenous.

I’m experimenting a lot in the past few months with all of these approaches.
It’s very hard to predict what you can or cannot do with LLMs because their output is completely contextual : it literally depends on 1. coding harness, 2. system prompt, 3. your codebase, 4. your request’s wording.

Personally, I share @Jappie 's confusion and excitement; what are the work, and the profession writ large turning into?

5 Likes

Do you have examples of this?

I find this a less interesting use case, but maybe it’s just me. I don’t design things I have no clue about, so the difficulty isn’t so much about coming up with ideas, but about making decisions.

And when it’s about making decisions, I prefer talking to humans with real experience. It’s also more fun.

Their output is chaotic in a mathematical sense. Sometimes rewording a question with no additional information causes wildly different results.

I think that mostly depends on how people use it. My suspicion is that the average LLM user isn’t driven by curiosity and mindful experimentation like perhaps you are.

Many of us have been deeply frustrated by:

  • AI companies shoving their tools into our face at every opportunity
  • employers starting to encourage or sometimes even demand AI use
  • poorly generated patches/documentation that I have to review even more carefully
  • a general lack of sensitivity towards the limitations of those tools
  • a decline in human collaboration
  • everything being about ā€œproductivityā€

I think it’s very well possible that it’s going to be a net negative. Maybe that also depends on what behavior we encourage/enable as an engineering community.

15 Likes

Just the titles of some recent interactions with Gemini:

In most cases I only had a passing knowledge of the topic, and I used the chat to gain new keywords and deciding whether a hunch is worth pursuing or not; takes 10 seconds.

I also prompt Claude to generate a shortlist of tradeoffs about a certain code design, and study that; I find this mostly useful.

We agree ! Still, most of these objections are economic in nature, and I’m arguing to take a ā€œglass half fullā€ stance.

1 Like

I think it very much does, since it is a significant aspect of the technology. I think people have to finally learn that the term ā€œtechnical argumentā€ is used to weasel out of responsibility. It is a political statement to not conflate ā€œtechnicalā€ with social, economic and cognitive dimensions.

You can run an LLM on your desktop workstation, in particular one trained with fully open and/or synthetic data.

I would love to see the link to the useful LLM which all the programmers are using that has been trained without labour exploitation and that I can run on my own machine. I would even buy a graphics card for that!

Mind you though, I think this is a strawman. It’s not about the theoretical possibility of an ethical LLM existing but about the reality that we’re confronted with right now:

  • people are buying the services from two companies
  • they are absolutely horrendous for the environment
  • and built on large scale exploitation

As for the shifting of labor bargaining power to capital, that didn’t start in 2023 with ChatGPT as I’m sure you’re aware.

This is literally we-should-improve-society-somewhat.jpg. I think it’s not a very good argument to say ā€œoh, bad things happened in the past so this new very bad thing is defensibleā€. You’re basically acknowledging that it’s not great and then you try to rationalize it immediately. This may be good for your conscience in the short term but it’s not very sustainable…

11 Likes

Trying to engage in good faith with zealotry usually ends up in disappointment, but despite your personal attacks I’ll try nonetheless.

I really don’t understand where you’re trying to get at. Which one is it?

  • coders/knowledge workers shouldn’t use LLMs
  • LLMs shouldn’t exist
  • we should forget LLMs ever existed
  • everyone and everything who uses LLMs is tainted by the original sin of labor exploitation and resource use

My point is that understanding this stuff by using it is also the only way to integrate it productively into society. Framing its denial as a moral purity battle is not.

5 Likes

At least link them to https://406.fail/ :smiley:

10 Likes

This is a piece of art :laughing:

Very well said @ocramz. Thanks for adding nuance. I’m experiencing the same.

2 Likes