Sometimes when I refactor, I do this manually with an LLM. It is useful in at least two ways: it can reveal better (more canonical) terminology for names (eg: 'antiparallel_line' instead of 'parallel_line_opposite_direction'), and it can also reveal names that could be generalized (eg: 'find_instance_in_list' instead of 'find_animal_instance_in_animals').

Edit: I'm currently trying it with a mere 1.2k JS file (openai mode) it's only 70% done after 20 minutes. Even if it works therodically with 50k LOC file, I don't think you should try.

I'm currently working on parallelizing the rename process, which should give orders of magnitude faster processing times for large files.

> Large files may take some time to process and use a lot of tokens if you use ChatGPT. For a rough estimate, the tool takes about 2 tokens per character to process a file:

> echo "$((2 * $(wc -c < yourscript.min.js)))" > So for refrence: a minified bootstrap.min.js would take about $0.5 to un-minify using ChatGPT.

> Using humanify local is of course free, but may take more time, be less accurate and not possible with your existing hardware.

Asking it to do it on the whole thing, then parsing the output and checking that the AST still matches?

1. It asks the LLM to write a description of what the variable does

2. It asks for a good variable name based on the description from 1.

3. It uses a custom Babel plugin to do a scope-aware rename

This way the LLM only decides the name, but the actual renaming is done with traditional and reliable tools.

Based on it, I went and read the readme. The readme was also excellent, and answered every question I had. Great job, thank you, I'll be trying this.

    for (var index = 0; index < inputLength; index += chunkSize) {

Making the code, fully commenting it and also giving an example after that might cost three times as much

Knowing this raises the question, which is better to feed an LLM source code of ASTs?

The answer is really it depends on the use case, there are tradeoffs. For example keeping comments intact possibly gives the model hints to reason better. On the other side, it can be argued that a pure AST has less noise for the model to be confused by.

There are other tradeoffs as well. For example, any analysis relating to coding styles would require the full source code.

Babel first parses the code to AST, and for each variable the tool:

1. Gets the variable name and surrounding scope as code

2. Asks the LLM to come up with a good name for the given variable name, by looking at the scope where the variable is

3. Uses Babel to make the context-aware rename to AST based on the LLM's response

For small scripts I've found the output to be very similar between small local models and GPT-4o (judging by a human eye).

API access of ChatGPT mode is needed as there are many round trips and it uses advanced API-only tricks to force the LLM output.

https://arxiv.org/pdf/2405.15793

It uses smart feedback to fix the code when LLMs occasionally do hiccups with the code. You could also have a "supervisor LLM" that asserts that the resulting code matches the specification, and gives feedback if it doesn't.

+1

A more general unminification or unobfuscation still seems to be an open problem. I wrote handful of programs that are intentionally obfuscated in the past and ChatGPT couldn't understand them even at the surface level in my experience. For example, a gist for my 160-byte-long Brainfuck interpreter in C had some comment trying to use GPT-4 to explain the code [1], but the "clarified version" bore zero similarity with the original code...

[1] https://gist.github.com/lifthrasiir/596667#gistcomment-47512...

Just because a task is simple doesn't mean its inverse need be. Examples:

  - multiplication / prime factorization
  - deriving / integrating
  - remembering the past / predicting the future

  - bugs
  - unused or irrelevant routines
  - incorrect implementations that incidentally give the right results

- Whitespace removal, which is trivially invertible.

- Comment removal, which we never expect to recover via unminification.

- Renaming to shorter names, which is tedious to track but still mechanical. And most minifiers have little understanding of underlying types anyway, so they are usually very conservative and rarely reuse the same mangled identifier for multiple uses. (Google Closure Compiler is a significant counterexample here, but it is also known to be much slower.)

- Constant folding and inlining, which is annoying but can be still tracked. Again, most minifiers are limited in their reasoning to do extensive constant folding and inlining.

- Language-specific transformations, like turning `a; b; c;` into `a, b, c;` and `if (a) b;` into `a && b;` whenever possible. They will be hard to understand if you don't know in advance, but there aren't too many of them anyway.

As a result, minified code still remains comparably human-readable with some note taking and perseverance. And since these transformations are mostly local, I would expect LLMs can pick them up by their own as well.

(But why? Because I do inspect such programs fairly regularly, for example for comments like https://news.ycombinator.com/item?id=39066262)

Let me give some concrete example from my old comment [1]. The full code in question was as follows, with only whitespaces added:

    function smb(){
      var a,b,c,d,e,h,l;
      return t(function(m){
        a=new aj;
        b=document.createElement("ytd-player");
        try{
          document.body.prepend(b)
        }catch(p){
          return m.return(4)
        }
        c=function(){
          b.parentElement&&b.parentElement.removeChild(b)
        };
        0

A nice, constrained, way to use a LLM here to enhance this solution is to ask it some variation of "what should this function be named?" and feed the output to a rename refactoring function.

You could do the same for variables, or be more holistic and ask it to rename variables and add comments (but risk the LLM changing what the code does).

This thread is incredibly distracting, at least 4 screenfuls to get through.

I'm really tired of the motte/bailey comments on HN on AI, where the motte is "meh the AI is useless, amateurish answer thats easy to beat" and bailey is "but it didn't name a couple global variables '''correctly'''." It verges on trolling at this point, and is at best self-absorbed and making the rest of us deal with it.

At least some of the time, simply taking it and reformatting to be unfolded and on multiple lines is useful enough to be readable/debuggable. FIXING that bug is likely more complex, because you have to find where it is in the original code, which, to my eyes, isn't always easy to spot.

ChatGPT is quite good at adding meaningful comments back to uncommented code, actually.

Paste some code and add "comment the shit out of this" as a prompt.

Minification does tend to obfuscate as as side effect, but it is not the goal, so reversing minification becomes much easier. Obfuscation on the other hand can minify code, but crucially that isn't the place it starts from. As the goal is different between minificaiton and obfuscation reversing them takes different efforts and I'd much rather attempt to reverse minification than I would obfuscation.

I'd also readily believe there are hundreds/thousands of examples online of reverse code minification (or here is code X, here is code X _after_ minifcation) that LLMs have ingested in their training data.

This is stated as if it's a truism, but I can't understand how you can actually believe this. Converting `let userSignedInTimestamp = new Date()` to `let x = new Date()` is trivial, but going the other way probably requires reading and understanding the rest of the surrounding code to see in what contexts `x` is being used. Also, the rest of the code is also minified, making this even more challenging. Even if you do all that right, it's at best it's still a lossy conversion, since the name of the variable could capture characteristics that aren't explicitly outlined in the code at all.

EDIT: I think I got why some comments complain I downplayed the power of LLM here. I never meant to, and I wanted to say that the unminification is a relatively easy task compared to other reverse engineering tasks. It is great we can automate the easy task, but we still have to wait for a better model to do much more.

Minification works in the same way. A lot of information needed for understanding the code is lost. Getting back that information can be a very demanding task.

Also, it should be noted that the name reconstruction is not a new problem and was already partly solved multiple times before the LLM era. LLM is great in that it can do this without massive retraining, but the reconstruction depends much on the local context (which was how earlier solutions approached the problem), so it doesn't really show its reasoning capability.

Of course, they didn't invent Generative pretraining (GP) or transfomers (T) but AFAIK they were the first to publicly combine them

every town and every street had "XEROX shops" where people went to get various documents photocopied for INR 1 per page for example

Most photocopy centers are still called XEROX Shops -- and their boards say that in big bold text: https://www.google.com/search?q=xerox+shop+india&udm=2

It doesnt matter if they use Canon, HP, or other brands of machines

(This is actually really interesting, I had no idea that 'hoover' was specifically a U.K. thing that didn't make it to the U.S.)

The relevant claim would be: all humans can do is move around in their environments, adapt the world around them through action, observe using adaptive sensory motor systems, grow and adapt their brains and bodies in response to novel and changing environments, abstract sensory motor techniques into symbolic concepts, vocalize this using inherited systems of meaning acquired as very young children in adaption within their environments, etc.

In the case of transformers all they can do is, in fact, sample from a compression of historical texts using a weighted probability metric.

If you project both of these into "problems an office worker has"-space, then they can appear simimlar -- but this projection is an incredibly dumb one, and offered as a sales pitch by charlatans looking to pretend that a system which can generate office emails can communicate.

This makes all "studies" whose aim is to approximate a fully representable abstract mathematical domain irrelevant to the question.

This is just more evidence of the naivety, mendacity, and pseudoscientific basis of ML and its research.

The structure of a mountain is not a pattern of pixels. So there is no function for a statistical alg to approximate, no n->infinity which makes the approximation exact.

By sampling from historical pixel patterns in previous images you can generate images in a pixel order that makes sense to a person already acquainted with what they represent. Eg., having seen a mountain (, having perspective, colour vision, depth, counterfactual simulation, imagination, ...).

In all these disagreeably dumb research papers that come out showing "world models" and the like you have the bad mathematicians and bad programmers called "AI researchers" giving a function approximation alg an abstract mathematical domain to approximate.

ie., if the goal is to "learn a circle" and you sample points from a circle, your approximation becomes exact in n->inf, because the target is *ABSTRACT*.

It's so dumb its kinda incomprehensible. It shows what a profound lack of understanding of science is rampent across the discipline.

MNIST, Games, Chess, Circles, Rulesets, etc. are all mathematical objects (shapes, rules). It is trivial to find a mathematical approximation to a mathematical object.

The world is not made out of pixels. Models of pixel patterns are not their targets.

I don't think that's all they can do.

I think they know more than what is explicitly stated in their training sets.

They can generalize knowledge and generalize relationships between the concepts that are in the training sets.

They're currently mediocre at it, but the results we observe from SOTA generative models are not explainable without accepting that they can create an internal model of the world that's more than just a decompression algorithm.

I'm going to step away from LLMs for a moment, but: How are video generator models capable of creating videos with accurate shadows and lighting that is consistent in the entire frame and consistent between frames?

You can't do that simply by taking a weighted average of the sections of videos you've seen in your training set.

You need to create an internal 3D model of the objects in the scene, and their relative positions in space across the length of the video. And no one told the model explicitly how to do that, it learned to do it "on its own".

I think the same principle applies to LLMs.

Compression is understanding. If you have a model which explains shadows you can compress your video data much better. Since you "understand" how shadows work.

You seem to think LLMs operate independently from humans. That doesn't happen in practice. We prompt LLMs, they don't just sample at random. We teach them new skills, share media and stories with them, work, learn and play together. It's not LLMs alone. They are pulled outside their training distribution by the user. The user brings their own unique life experience into the interaction.

However, I do think that in this case it's meaningful. The claim isn't that LLMs are genuinely exhibiting reasoning ability — I think it's quite clear to anyone who probes them for long enough that they're not. I was fooled initially too, but you soon come to realise it's a clever trick (albeit not one contrived by any of the human designers themselves). The claim is usually some pseudo-philosophical claim that the very definition of reasoning is simply 'outputting (at least some of the time) correct sentences' and so there's no more to be said. But this is just silly. It's quite obvious that being able to manipulate language and effectively have access to a vast (fuzzily encoded) database of knowledge will mean you can output true and pertinent statements a lot of the time. But this doesn't require reasoning at all.

Note that I'm not claiming that LLMs exhibit reasoning and other abilities 'as a side effect' of language manipulation ability — I'm claiming there's no reason to believe they have these abilities at all based on the available evidence. Humans are just very easily convinced by beings that seem to speak our language and are overly inclined to attribute all sorts of desires, internal thought processes and whatever else for which there are no evidence.

this overlooks how they do it. we don't really know. it might be logical reasoning, it might be a very efficient content addressable human-knowledge-in-a-blob-of-numbers lookup table... it doesn't matter if they work, which they do, sometimes scarily well. dismissing their abilities because they 'don't reason' is missing the forest for the trees in that they'd be capable of reasoning if they were able to run sat solvers on their output mid generation.

I think most people talking past each other are really discussing whether the GPT is conscious, has a mental model of self, that kind of thing, as long as your definition of reasoning doesn't include consciousness it clearly does it (though not well.)

Also: what's his rationale? It's no use simply claiming something without evidence. And as far as I (and seemingly most others) can see, there's no such evidence other than that they can sometimes output sentences that happen to be true. But so can Wikipedia — does that mean Wikipedia is reasoning?

Also, any form of reasoning in the usual sense of the word would surely require the ability to allocate arbitrary amounts of computation (i.e. thought) to each question. LLMs don't do this — they don't sit and ponder; each token takes exactly the same amount of computation to produce. Once they hit an 'end of text' token, they're done.

Even empirically speaking, LLMs' ability to reason can be seen to be nonexistent. Just try asking basic mathematics questions. As soon as you ask anything for which the answer isn't available — practically verbatim — on the web already, it produces intelligent-sounding gibberish.

This whole idea that 'LLMs must be able to reason because in order to learn to fake reasoning you must learn to actually reason' is like some kind of inverted no true Scotsman fallacy.

Yes, Hinton can be wrong, is wrong on many things like his misunderstanding on Chomsky and language.

But I also think he has spent thousands of hours testing these systems scientifically.

Your last sentence puts a lot of words in peoples mouths. But to continue down that line, fake reasoning and actual reasoning sounds like the Chinese Room. Is that the argument you are making?

We don't understand our own mental processes well enough, so I try to not anthropomorphize reasoning and cognition.

Well, it’s the most common sentiment I see on both here and (before I gave up) the AI-centred parts of reddit.

It’s not quite the Chinese Room, since LLMs can’t even simulate reasoning very well. So there’s no need to debate the distinction between ‘fake reasoning and actual reasoning’ — there may or may not be a difference, but it’s not the point I’m making.

As for Hinton: I’m sure he has. But inventors are often not experts on their own creations/discoveries, and are probably just as prone to FUD and panic in the face of surprising developments as the rest of us. No one predicted that autoregressive transformers would get us this far, least of all the experts whose decades of work lead us to this point.

They can’t sort a list but they can translate languages, for instance, given that a list sorted almost right is wrong but that we will often settle for an almost right translation.

I tried translating a python script to javascript the other day and it was flawless. I would expect it to scale with a bit of hand-railing.

It does a well at writing simple to medium complexity automation scripts around AWS.

If it gets something wrong, I tell it to “verify your answer using the documentation available on the web”

It was scary to me how to chatting with GPT or Claude would give me information which was a lot more clear than what I could deduce after hours of reading AWS documentation.

Perhaps, the true successor to Google search has arrived. One big drawback of Google was asking questions that can't be converted to a full long conversation.

To that end. LLM chat is the ultimate socratic learning method tool till date.

It's nothing short of incredible. Each of those tasks would normally have taken me hours and I have working code in actual seconds.

As IDE integration grows and there are more and better models, that can do this better than ever. We will unlock all sort of productivity benefits.

There is still skepticism about making these work at scale, with regards to both electricity and compute requirement for the larger audience. But if they can get this to work, we might see a new era tech boom way bigger than we have seen anything before.

Reminds me of the tool that was provided in older versions of ColdFusion that would "encrypt" your code. It was a very weak algorithm, and didn't take long for someone to write a decrypter. Nevertheless some people didn't like this, because they were using this tool, thinking it was safe for selling their code without giving access to source. (In the late 90s/early 2000s before open source was the overwhelming default)

Website offers some kind of contest which is partly dependent on obfuscated client side code.

Clever contestants can now through it into ChatGPT to improve their chances.

Now, it begins.

There’s no denying it. This task is intellectual. Does not involve rote memorization. There are not tons and tons of data pairs on the web of minimized code and unminified code for llms to learn from.

The llm understands what it is unminifying and it is in general superior to humans on this regard. But only in this specific subject.

> There are not tons and tons of data pairs on the web of minimized code and unminified code for llms to learn from.

Are you sure about this? These can be easily generated from existing JS to use as a training set, not to mention the enormous amount of non-minified JS which is already used to train it.

The challenge of understanding minified code for a human comes from opaque variable names, awkward loops, minimal whitespacing, etc. These aren't things that a computer has trouble with: it's why we minify in the first place. Attention, as a scheme, should do great with it.

I'd also say there is tons of minified/non-minified code out there. That's the goal of a map file. Given that OpenAI has specifically invested in web browsing and software development, I wouldn't be surprised if part of their training involved minified/unminified data.

They are irrelevant for executing the code, but they're probably pretty relevant for an LLM that is ingesting the code and text and inferring its function based on other examples it has seen. It's definitely more impressive that an LLM can succeed at this without the context of (correct) variable names than with them.

There was a time when winning in Chess was a proof of humans' superior intellect, and then it became just an algorithm. Then Go.

GPT-4 has consumed more code than your entire lineage ever will and understands the inherent patterns between code and minified versions. Recognizing the abstract shape of code sans variable names and mapping in some human readable variable names from a similar pattern you've consumed from the vast internet doesn't seem farfetched.

Having a computer multiplying 1000-digit numbers instantly is an example of humans succeeding at multiplying: by creating a tool for that first. Because what else is intellectually succeeding there? It’s not like the computer has created itself.

If one draws a boundary of human intelligence at the skull bone and does not count the tools that this very intelligence is creating and using as mere steps of problem solving process, then one will also have to accept that humans are not intelligent enough to fly into space or do surgery or even cook most of the meals.

LLMs are very good at a surprisingly broad array of semi-structured-text-to-semi-structured-text transformations, particularly within the manifold of text that is widely available on the internet.

It just so happens that lots of code is widely available on the internet, so LLMs tend to outperform on coding tasks. There’s also lots of marketing copy, general “encyclopedic” knowledge, news, human commentary, and entertainment artifacts (scripts, lyrics, etc). LLMs traverse those spaces handily as well. The capabilities of AI ultimately boil down to their underlying dataset and its quality.

I'm simply saying the AI has superior performance to humans on this specific subject. That's all.

Why did you suddenly make this comment of "bowing before your god" when I didn't even mention anything remotely close to that?

I'll tell you why. Because this didn't come from me. It came from YOU. This is what YOU fear most. This is what YOU think about. And your fear of this is what blinds you to the truth.

After reading through this article, I tried again [0]. It gave me something to understand, though it's obfuscated enough to essentially eval unreadable strings (via the Window object), so it's not enough on it's own.

Here was an excerpt of the report I sent to the person:

> For what it’s worth, I dug through the heavily obfuscated JavaScript code and was able to decipher logic that it:

> - Listens for a page load

> - Invokes a facade of calculations which are in theory constant

> - Redirects the page to a malicious site (unk or something)

[0] https://chatgpt.com/share/f51fbd50-8df0-49e9-86ef-fc972bca6b...

It has been incredibly liberating to just feed it a spaghetti mess, ask to detangle it in a more readable way and go from there.

As the author also discovered, LLMs will sometimes miss some details, but that is alright as I will be catching those myself.

Another use case is when I understand what the code does, but can't quite wrap my head around why it is done in that specific way. Specifically, where the author of the code is no longer with the company. I will then simply put the method in the LLM chat, explain what it does, and just ask it why some things might be done in a specific way.

Again, it isn't always perfect, but more often than not it comes with explanations that actually make sense, hold up under scrutiny and give me new insights. It actually has prevented me once or twice from refactoring something in a way that would have caught me headaches down the line.

[0] chatGPT and more recently openwebUI as a front end to various other models (Claude variants mostly) to see the differences. Also allows for some fun concepts of having different models review each others answers.

[ed.: looks like this was an encoding problem, cf. thread below. I'm still a little concerned about correctness though.]

    // ORIGINAL:
    j.useEffect(() => {
        function r() {
            n({ height: window.innerHeight, width: window.innerWidth });
        }
        if (typeof window < "u") return n({ height: window.innerHeight, width: window.innerWidth }), window.addEventListener("resize", r), () => window.removeEventListener("resize", r);
    }, []),

    // UNMINIFIED:
    useEffect(() => {
      const handleResize = () => {
        setSize({ height: window.innerHeight, width: window.innerWidth });
      };

      // Initial size setting
      handleResize();

      window.addEventListener('resize', handleResize);
      return () => {
        window.removeEventListener('resize', handleResize);
      };
    }, []);

Discrediting the entire output because a few missing characters would be very pedantic.

Otherwise, the output is identical as far as I can tell by looking at it.

I updated the article to reflect the mistake.

> Update (2024-08-29): Initially, I thought that the LLM didn’t replicate the logic accurately because the output was missing a few characters visible in the original component (e.g., ). However, a user on HN forum pointed out that it was likely a copy-paste error.

>

> Upon further investigation, I discovered that the original code contains different characters than what I pasted into ChatGPT. This appears to be an encoding issue, as I was able to get the correct characters after downloading the script. After updating the code to use the correct characters, the output is now identical to the original component.

>

> I apologize, GPT-4, for mistakenly accusing you of making mistakes.

The web pre-dates utf-8, although not by much. Ken Thompson introduced utf-8 at winter Usenix in 1993 and CERN released the web in April, but it would be several more years before utf-8 became common. The early web was ISO 8859-1 by default. But people were pretty lazy about specifying character sets back then (still are actually) and Microsoft started sending or assuming their 1252 character set where 8859-1 was required by the spec. Eventually the spec was changed to match de facto behavior. I guess the assumption was that if you're too stupid or lazy to say what character set you're using, then it's probably 1252. (Today the assumption would be that it's probably utf-8). I'm not sure what the specs say today, but I think html is assumed to be in utf-8, and everything else is assumed to be 1252 (if the character set is not explicitly declared).

I would guess if he just told it to rename the variables and method first, it would have been closer to the original.

All this makes me think AI's are going to be a strong deflationary force in the future.

>If an LLM can interact with a browser and figure out how a UI works and take code hints from un-mimified code I think we could see all SAAS apps be commoditized. The backend would be proprietary, but it could figure out API formats and suggest a backend architecture.

whoooha! that's a lot of probing and testing of the SAAS that would be required in order to see how it behaved. SAAS aren't algorithms, they operate over data that's unseen on the front end as well...

>All this makes me think AI's are going to be a strong deflationary force in the future.

I don't get this. I've literally never worked anywhere which had enough software engineers, we've been going on about software crisis for about 50 years and things are arguably worse than ever. The gap between the demand for good software (in the sense that allocating capital to producing it would be sensible) and the fulfillment of that demand is bigger than ever. We just don't have the mechanisms to make this work and to make it work at an economically viable level.

Then we get AI to help us and everyone thinks that the economy will shrink?

Someone could already easily clone HubSpot relatively cheaply even if they hired developers, but that doesn't mean it will be anywhere near successful.

Actually this opens up a bigger question. What if I like an open source project but don't like its license. I can just prompt AI by giving it the open source code and ask it to rewrite it or write in some other language. Have to look up the rules if this is allowed or will be considered copying and how will a judge prove?

In the historic Google v. Oracle suit, the only actual code that was claimed to be copied was a trivial `rangeCheck` function, but Google's intent and other circumstances like the identical code structure and documentation made it much more complicated, and the final decision completely bypassed the copyrightability of APIs possibly for this reason.

I am testing large language models against a ground truth data set we created internally. Quite often when there is a mismatch, I realize the ground truth dataset is wrong, and I feel exactly like the author did.

edit: chatgpt found out that its rot13 and couldn't explain the code directly without deobfuscating it first.

> $ ask4 ' what does dGhhdCBpcyBxdWl0ZSBpbnRlcmVzdGluZw== decode to? ' > A "dGhhdCBpcyBxdWl0ZSBpbnRlcmVzdGluZw==" is a Base64 encoded string. When decoded, It translates to "that is quite interesting" in English.

Is it? It’s probably more obscuring from an LLM’s perspective, assuming the LLM has seen enough rot13 text during training. Spaces and punctuation are untouched by rot13, unlike base64, which means that word and sentence boundaries will still be denoted by tokens that denote those boundaries in plaintext.

However I can confirm that Claude was able to identify that it's rot13 and also respond properly.

> The provided code is quite complex, but I'll break it down into a more understandable format, explaining its different parts and their functionalities.

Reading the above statement generated by ChatGPT, I asked myself: Will we live to the day where these LLMs could take a large binary executable as input, read it, analyze it, understand it, then reply with the above statement?

> I followed up asking to "implement equivalent code in TypeScript and make it human readable" and got the following response.. To my surprise, the response is not only good enough, but it is also very readable.

What if this day came and we can ask these LLMs to rewrite the binary code in [almost] any programming language we want? This would be exciting, yet scary to just think about!

I have been doing the Leetcode thing recently, and even became a subscriber to Leetcode.

What I have been doing is I go through the Grind 75 list (Blind 75 successor list), look for the best big O time and space editorial answer, which often has a Java example, and then go to ChatGPT (I subscribe) or Perplexity (don't subscribe to Pro - yet) and say "convert this to Kotlin", which is the language I know best. Jetbrains IDE or Android Studio is capable of doing this, but Perplexity and ChatGPT are usually capable of doing this as well.

Then I say "make this code more compact". Usually I give it some constraints too - keep the big O space and time complexity the same or lower it, keep the function signature of the assigned function the same, and keep the return explicit, make sure no Kotlin non-null assertions crop up. Sometimes I continually have it run these instructions on each version of the iterated code.

I usually test that the code compiles and returns the correct answers for examples after each iteration of compacting. I also copy answers from one to the other - Perplexity to ChatGPT and then back to Perplexity. The code does not always compile, or give the right answers for the examples. Sometimes I overcompact it - what is clear in four lines becomes too confusing in three compacted lines. I'm not looking for the most compact answer, but a clear answer that is as compact as possible.

One question asked about Strings and then later said, what if this was Unicode? So now for String manipulation questions I say assume the String is Unicode, and then at the end say show the answer for ASCII or Unicode. Sometimes the big O time is tricky - it is time O(m+n) say, but since m is always equal to or less than m in the program, it is actually O(n), and both Perplexity and ChatGPT can miss that until it is explained.

People bemoan Leetcode as a waste of time, but I am wasting even less time with it, as ChatGPT and Perplexity are helping give me the code I will be demonstrating in interviews. The common advice I have heard from everywhere is don't waste time trying to figure out the answers myself - just look at the given answers, learn them, and then look for patterns (like binary search problems, which are usually similar), so that is what I am doing.

Initially I was a ChatGPT and Perplexity skeptic for early versions of those sites, in terms of programming, as they stumbled more, but these self-contained examples and procedures they seem well-suited for. Not that they don't hallucinate or give programs that don't compile, or give the wrong answers sometimes, but it saves me time ultimately.

However, I wonder about this: What will you do in a live interview situation? Will you pull up ChatGPT?

As a next step - even within these 75 questions, Grind 75's eighth answer and fourteenth answer are answered essentially the same way, as are other questions in there. So the next step would be to see these patterns (binary search, priority queues, sliding window, backtracking) and how to answer them, and then be able to solve them in slightly novel problems (in the more complex questions I understand one might run into more than one of these patterns).

Train on java compiled to class files. Then go from class back to java.

Or even:

Train java compiled to class files, and have separate models that train from Clojure to class and Scala to class files. Then see if you can find some crufty (but important) old java project and go: crufty java -> class -> Clojure (or Scala).

If you could do the same with source -> machine instructions, maybe COBAL to C++! or whatever.

LLM source recovery from binaries is thing. The amazing part is that they are pretty good at adding back meaningful variable names to the generated source code.

AI cannot even lint properly right now and you want it to decompile? good luck, there's too much hype going on people really think this is possible this year?

In the end always remember it's just autocomplete, it's pretty terrible at translations that are not natural language to natural language. I worked on a natural language to SQL and it was impossible to make it consistently generate valid SQL for Postgres, and I'm talking about natural language to SQL not virtual machine instructions...

Really all this rule amounts to is

    // Increment by a penny
    accountValue += 1

    // Parse the filename and remove the extension
    let fext_re = Regex::new(r"(.\*)\.(.+)$").unwrap();
    let page_cap = fext_re.captures(fname).unwrap();
    let page_base_filename = page_cap.get(1).unwrap().as_str();

To boot, a unit test could be written that would reveal the bug in the regular expression that makes it only work with filenames that have an asterisk before the extension. Unless you intended that (unlikely), in which case the comment is wrong/not comprehensive and misdirects the reader.

A comment that is incorrect can do a lot of damage, and they tend to get confused about implementation details over time.

Not to a competent programmer when reading well-written code.

This also means that you read what the code does, rather than what a comment says the code does. Otherwise you will be blind to bugs. Any experienced developer will tell you that code very often doesn't do what the original programmer thought it did.

No, literally reading a one line about what the next 4 lines do is mechanically faster. It does not matter that you are good or bad, it is about simple reading speed.

> This also means that you read what the code does, rather than what a comment says the code does. Otherwise you will be blind to bugs. Any experienced developer will tell you that code very often doesn't do what the original programmer thought it did.

I am an experience developer. I have worked on several "legacy" projects, and started many from 0.

1. It does not make you blind to anything, it is just a way to learn/direct yourself in the code base faster.

2. Knowing what the original developer wanted is often as useful as knowing what the code actually does. More info is better than no info.

Even outdated comment can be useful.

For me, this type of thinking that comment are unnecessary, that competent ppl can just read the code, etc. is actually a sign of younger dev who never had to work on a long-lived codebase.

It sounds like you're conflating "helpful comments that explain why" with "no comments are needed ever because read the code", and we're talking past each other.

For example, "Here's my app.js file, please add an endpoint for one user to block another. Feel free to suggest schema changes. Please show me the full app.js with these changes implemented"

The model seems to be great at figuring out frameworks and databases just by seeing the contents of a full app.js file.

I do find this type of prompt works much better with Sonnet3.5 than GPT4o.

I noticed while reading the blog entry that the author described using a search engine multiple times and thought, "I would have asked ChatGPT first for that."

Training data would be easy to make in this case. Build tons of free GitHub code with various compilers and train on inverting compilation. This is a case where synthetic training data is appropriate and quite easy to generate.

You could train the decompiler to just invert compilation and the use existing larger code LLMs to do things like add comments.

For legal reasons I bet this will become blocked behavior in major models.

Unfortunately not really. Having the source is a first step, but you also need the rights to use it (read, modify, execute, redistribute the modifications), and only the authors of the code can grant these rights.

Or, going back to the original idea, while the source code produced in such a way might be illegal, it's very likely 'clean' enough to train an LLM on it to be able to help in reproducing such an application.

I would guess clean room would still require having someone reading the LLM-decompiled code, write a spec, and have someone else write the code.

But this is definitely a good question, especially given the recent court verdicts. If you can launder open source licensed code, why not proprietary binaries? Although I don't think the situation is the same. I wouldn't expect how you decompile a code matters.

I agree. I think "AI generating/understanding source code" is a huge red herring. If AI was any good at understanding code, it would just build (or fix) the binary.

And I believe how it will turn out to be, when we really have AI programmers, they will not bother with human-readable code, but code everything in machine code (and if they are tasked in maintaining existing system, they will understand in its entirety, across the SW and HW stack). It's kinda like diffusion models that generate images don't actually bother with learning drawing techniques.

If anything I expect AI-written programs in the not so distant future to be incomprehensible because they're too short. Something like reading an APL program.

In practice, compilation is often (not always, agreed!) from a language A to a lower level language B such that the runtime for language A can't run language B or vice-versa, if language A has a runtime at all. Minification is always from language A to the same language A.

The implication is that in practice, deminification is not the same exercise as decompilation. You can even want to run a deminification phase after a decompilation phase, using two separate tools, because one tool will be good at translating back, and the other will be good at pretty printing.

[1] https://en.wikipedia.org/wiki/Reductionism

I usually crap on things like chatgpt for being unreliable and hallucinating a lot. But in this particular case, decompilers already usually generate inaccurate code, and it takes a lot of work to fix the decompiled code to make it correct (I speak from experience). So introducing AI here may not be such a huge stretch. Just don't expect an AI/LLM to generate perfectly correct decompiled code and we're good (wishful thinking).

That's not how copyright and licensing works.

You could already break the law and open yourself up to lawsuits and prosecution by stealing intellectual property and violating its owners rights before there were LLMs. They just make it more convenient, not less illegal.