https://news.ycombinator.com/item?id=38194107

alwayshasbeen.jpg

There have been articles about how "data is the new oil" for a couple of decades now, with the first reference I could find being from British mathematician Clive Humby in 2006 [0]. The fact that it rings even more true in the age of LLMs is simply just another transformation of the fundamental data underneath.

[0] https://en.wikipedia.org/wiki/Clive_Humby#cite_ref-10

Bard's probably just a middle man here.

The response: "I'm a text-based AI, and that is outside of my capabilities."

For me it returns a seemingly accurate answer [1], albeit missing his involvement with Twitter/X. But LLMs are intrinsically stochastic, so YMMV.

[1] https://g.co/bard/share/378c65b56aea

Another interesting line of inquiry (potentially revealing some biases) is to ask it whether someone is a supervillain.

Personally, I struggle with anything even slightly technical from all of the current LLM's. You really have to know enough about the topic to detect BS when you see it... which is a significant problem for those using it as a learning tool.

I just ignore how confident ChatGPT sounds.

I don't know the name for the effect, but it's similar to when you listen/watch the news. When the news is about a topic you know an awful lot about, it's plainly obvious how wrong they are. Yet... when you know little about the topic, you just trust what you hear even though they're as likely to be wrong about that topic as well.

The problem is people (myself included) try to use GPT as a guided research/learning tool, but it's filled with constant BS. When you don't know much about the topic, you're not going to understand what is BS and what is not.

Obviously they need to fix that for realistic usage, but I use it as a studying technique. Usually when I ask it to give me some detailed information about stuff that I know a bit about, it will get some details about it wrong. Then I will argue with it until it admits that it was mistaken.

Why is this useful? Because it gets "just close enough to right" that it can be an excellent study technique. It forces me to think about why it's wrong, how to explain why it's wrong, and how to utilize research papers to get a better understanding.

> OpenAI’s technologies had the lowest rate, around 3 percent. Systems from Meta, which owns Facebook and Instagram, hovered around 5 percent. The Claude 2 system offered by Anthropic, an OpenAI rival also based in San Francisco, topped 8 percent. A Google system, Palm chat, had the highest rate at 27 percent.

https://www.nytimes.com/2023/11/06/technology/chatbots-hallu...

The main thing as a user is that they require different nudges to get the answer you are after out of them, i.e. different ways of asking or prompt eng'n

Honestly, $20/month is pretty cheap in my case; I feel like I definitely extract much more than $20 out of it every month, if only on the number of example stubs it gives me alone.

I've also noticed that API based clients (rather than the web or iOS client) result in conversations that hold my hand less. The voice client seems hopeless though, probably because I write ok, but have trouble saying what I want before the stupid thing cuts me off. It seems to love making lists, and ignoring what I want.

I think it's running some kind of heuristic on the output before passing it to the user, because slightly different prompts will sometimes succeed.

ChatGPT's system is smart enough to recognize that fantasy crimes are not serious information about committing real crimes or whatever.

For a fair comparison, you probably need to try while ChatGPT is working.

Actually at Google scale I wouldn't expect so

I actually had a discussion with Phind itself recently, in which I said that in order to help me, it seems like it would need to ingest my codebase so that it understands what I am talking about. Without knowing my various models, etc, I don't see how it could write anything but the most trivial functions.

It responded that, yes, it would need to ingest my codebase, but it couldn't.

It was fairly articulate and seemed to understand what I was saying.

So, how do people get value out of Phind? I just don't see how it can help with any case where your function takes or returns a non-trivial class as a parameter. And if can't do that, what is the point?

It is also capable to perform searches, which lead me - forgive me founders - to abuse it quite a lot: whenever I am not finding a good answer from other search engines I turn up to Phind even for things totally unrelated to software development, and it usually goes very well.

Sometimes I even ask it to summarize a post, or tell me what HN is talking about today.

I am very happy with it and hope so much it gains traction!

So if I create a GPT for my open-source library as a way to fund it, all these copilot etc. are going to compete with me?

Just wondering because that would be a bummer to not have this avenue to fund open-source code.

Nit: your link has a trailing "s" which makes it 404 :)

Also I use it for LaTeX, too. It is very helpful providing various package than trying to hunt more information through Google. I got a working tex file within 15 min than it took me 3 weeks 5 years ago!

As a whole I think it works well in tandem with ChatGPT to bounce ideas or get alternate perspectives.

(I also love the annotation feature where it shows the websites that it pulled the information from, very well done)

"The inference service may be temporarily unavailable - we have alerts for this and will be fixing it soon."

Is /s a self-fulfilling sarcasm indicator or a typo?

I'm still surprised by the problems with it. Last month it lied about some facts then claimed to have sent an email when asked for more details.[1]

Then apologized for claiming to send an email since it definitely did not and "knew" it could not.

It's like a friend who can't say 'I don't know' and just lies instead.

1. I was asking if the 'Christ the King' statue in Lisbon ever had a market in it, a rumor told to me by a local. It did not, contrary to Bard's belief.

:)

I had a bug that wouldn't let me login to my work OpenAI account at my new job 9 months ago. It took them 6 months to respond to my support request and they gave me a generic copy/paste answer that had nothing to do with my problem. We spend tons and tons of money with them and we could not get anyone to respond or get on a phone. I had to ask my coworkers to generate keys for everything. One day, about 8 months later, it just started working again out of nowhere.

We switched to Azure OpenAI Service right after that because OpenAI's platform is just so atrociously bad for any serious enterprise to work with.

I know they have money, but money isn't a magic wand for creating people. They could've also kept it a limited beta for much longer, but that would've killed their growth velocity.

So here is a great product that provides no SLA at all. And we all accept it, because having it most of the time is still better than having it not at all ever.

Your example is clearly not acceptable, but I can see reasons for it.

OpenAI apparently was somewhere between "I can't see people finding this useful" and "I guess" when deciding on releasing ChatGPT at all in the first place.

If that's the case, I doubt they were envisioning a flood of users, who needed a customer support person to handle their case. They have to spin-up an entire division to handle all of this. And I'm sure some of the use-cases are going to get into complex technical issues that might be hard to train people for.

They can no longer remain a heads-down company full of engineers working on AI.

I'm not excusing it, but I can see why things like your situation might occur. Although 6 months for a response is obviously ridiculous. If you are paying them a significant amount of money, and it is impacting your business, then that's all on OpenAI to fix ASAP.

I also cannot login on Firefox (latest version) with strict privacy settings and AdNauseam on desktop.. and a few weeks ago they broke their website on iOS v14 as well for no apparent reason (it certainly didn't make me to download their app since that require v16.1+).

Also, they need to remain flexible most likely in their infrastructure to make the changes.

As an architecture guy, I sense when the rate of change slows down more SLA type stuff will come up, or may be available first to Enterprise customers who will pay for the entire cost of it. Maybe over time there will be enough slack there to extend some SLA to general API users.

In the meantime, monitoring API's ourselves isn't that crazy. Great idea to use more than one service.

Basically, we use AI to do a lot of formatting for our manuals. It's most useful with the backend XML markups, not WYSIWYG editors.

So, we take the inputs from engineers and other stakeholders, essentially in email formats. Then we pass it through prompts that we've been working on for a while. Then it'll output working XML that we can use with a tad bit of clean-up (though that's been decreasing).

It's a lot more complicated than just that, of course, but that's the basics.

Also, it's been really nice to see these chat based AIs helping others code. Some of the manuals team is essentially illiterate when it comes to code. This time last year, they were at best able to use excel. Now, with the AIs, they're writing Python code of moderate complexity to do tasks for themselves and the team. None of it is by any means 'good' coding, it's total hacks. But it's really nice to see them come up to speed and get things done. To see the magic of coding manifest itself in, for example, 50 year old copy editors that never thought they were smart enough. The hand-holding nature of these AIs is just what they needed to make the jump.

It sounds like a pretty old and common use case in technical writing and one that many organizations already optimized plenty well: you coach contributors to aim towards a normal format in their email and you maintain some simple tooling to massage common mistakes towards that normal.

What prompted you to use an LLM for this instead of something more traditional? Hype? Unfamiliarity with other techniques? Being a new company and seeing this as a more compelling place to start? Something else?

Here's a session from me working on a side project yesterday:

https://chat.openai.com/share/a6928c16-1c18-4c08-ae02-82538d...

The most impressive thing I think starts in the middle:

* I paste in some SQL tables and the golang structrues I wanted stuff to go into, and described in words what I wanted; and it generated a multi-level query with several joins, and then some post-processing in golang to put it into the form I'd asked for.

* I say, "if you do X, you can use slices instead of a map", and it rewrites the post-processing to use slices instead of a map

* I say, "Can you rewrite the query in goqu, using these constants?" and it does.

I didn't take a record of it, but a few months ago I was doing some data analysis, and I pasted in a quite complex SQL query I'd written a year earlier (the last time I was doing this analysis), and said "Can you modify it to group all rows less than 1% of the total into a single row labelled 'Other'?" And the resulting query worked out of the box.

It's basically like having a coding minion.

Once there's a better interface for accessing and modifying your local files / buffers, I'm sure it will become even more useful.

EDIT: Oh, and Monday I asked, "This query is super slow; can you think of a way to make it faster?" And it said, "Query looks fine; do you have indexes on X Y and Z columns of the various tables?" I said, "No; can you write me SQL to add those indexes?" Then ran the SQL to create indexes, and the query went from taking >10 minutes to taking 2 seconds.

(As you can tell, I'm neither a web dev nor a database dev...)

I do. If too many of our apprentices don’t actually learn how to work the forge, how ready will they be to take over as masters someday themselves?

I can see how ChatGPT was useful to the grandparent today, but got very disturbed by what it might portend for tomorrow. Not because of job loss and automation, like many people worry, but because of spoiled training and practice opportunities.

I liked your take, so I’d be curious to hear what you think.

Looking forward to the Y2K levels of highly paid consulting work becoming available now you mention it

We've long since reached the point at which no one can be said to be a true polymath ( https://en.wikipedia.org/wiki/The_Last_Man_Who_Knew_Everythi... ). Having lost the ability as individuals to know something about everything, we're now losing the ability to know everything about anything.

I'm pretty sure that while the most popular programming languages today are Python and Javascript, the most popular ones 10 years from now will be English and Mandarin. Everything we know about software development is about to change. It's about time.

Lesser issues: additional strain on index rebuilding whenever that happens; messing with execution plans and causing the query planner to be inefficient; primary/secondary memory overhead; or if your DB engine uses locks you can run into a myriad of issues there.

I'm all for SWEs learning about databases, as I'm morally opposed to the proliferation of ORMs and the like, but I don't think ChatGPT is the right way to go about things long-term. It's similar to Googling or using StackOverflow: yes, you will find information that is relevant to what interests you at the moment, but it's soon forgotten and does nothing to help build long-term mental models.

It saved a bunch of manual work on a throwaway script. In the past, I might have done something in Python, since I'm more familiar with it than powershell. Or, I'd say, "well, it's only 20 files. I'll just do it manually." The GPT script worked on the first try, and I just threw it away at the end.

In the end I settled on a standalone desktop app to "compose" prompt with source code, instructions and formatting options which I can just copy paste into ChatGPT.

The app is available for download if anyone is interested: https://prompt.16x.engineer/

I also use it heavily for formatting adjustments. Instead of hand-formatting a transcript I pull from YouTube, I paste it into Claude and have it reformat the transcript into something more like paragraphs. Many otherwise tedious reformatting tasks can be simplified with an LLM.

I also will get an LLM to develop flashcards for a given set of notes to drill on, which is nice, though I usually have to heavily edit the output to include everything I think I should study.

In class, if I'm falling behind on notetaking, I'll get the LLM to generate the note I'm trying to write down by just asking it a basic question, like: "What is anarchism in a sentence?" That way I can focus on what the teacher is saying while the LLM keeps my notes relevant. I'll skim what it generates and edit to fit what my prof said, but it's nice because I can pay better attention than if I feel I have to keep track of what the prof might test me on. This actually is a note-taking technique I've learned about where you only write down the question and look up the answer later, but I think it's nice I now can do the lookup right there and tailor it to exactly how the prof is phrasing it/what they're focusing on about the topic.

What I'm about to say is in the context of programming. I have the tendency to get caught up in some trivial functionality, thus losing focus on the overall larger and greater objective.

If I need to create some trivial functionality, I start with unit tests and a stubbed out function (defining the shape of the input). I enumerate sufficient input/output test cases to provide context for what I want the function to do.

Then I ask copilot/ChatGPT to define the function's implementation. It sometimes takes time to tune the dialog or add some edge cases to the the test cases, but more often than not copilot comes through.

Then I'm back to focusing on the original objective. This has been a game changer for me.

(Of course you should be careful about what code is generated and what it's ultimately doing.)

It's a bit different from other plugins which only act on the text in the buffer in that it also sends the diagnostics from the LSP to ChatGPT too.

https://github.com/piersolenski/wtf.nvim

1. How many times a [day/month] do you use it?

2. In your experience how often does GPT 'hallucinate' an explanation?

Ask it to document the conditions according to the code and taking into consideration the following x, y, z.

Output a raw markdown table with the columns a, b, c.

Translate column a in English between ()

---

Speeds up the "document what you're doing" for management purpose, while I'm actually coding and testing out scenarios.

Tbh. I'm probably one of the few that did the coding while "doing the analysis".

Ps. It's also great for writing unit tests according to arrange, act, assert.

I wrote a couple commandline tools to do things like autogenerate commit comments or ask it questions from the commandline and return the right bash invocation to do whatever I need done https://github.com/pmarreck/dotfiles/blob/master/bin/functio...

Random thing I did this morning was see if it could come up with an inspiring speech to start healing the rift between israel and its neighbors https://chat.openai.com/share/71498f5f-3672-47cd-ad9a-154c3f...

It's very good at returning unbiased language

As for me, I’ve got other uses for $45k.

You can run (much) smaller LLM models on consumer-grade GPUs though. A single Nvidia GPU with 8 GB RAM is enough to get started with models like Zephyr, Mistral or Llama2 in their smallest versions (7B parameters). But it will be both slower and lower quality than anything OpenAI currently offers.

Longer: In theory, but it'll require a bunch of glue and using multiple models depending on the specific task you need help with. Some models are great at working with code but suck at literally anything else, so if you want it to be able to help you with "Do X with Y" you need to at least have two models, one that can reason up with an answer, and another to implement said answer.

There is no general-purpose ("FOSS") LLM that even come close to GPT4 at this point.

It’s probably as good as you can get at the moment though; and hey, trying it out costs you nothing but the time it takes to download llama.cpp and run “make” and then point it at the q6 model file.

So if it’s no good, you’ve probably wasted nothing more than like 30 min giving it a try.

[1] - https://huggingface.co/TheBloke/CodeLlama-34B-Instruct-GGUF [2] - https://github.com/ggerganov/llama.cpp

For my hand typed use case's, GPT-4 is the only acceptable model that doesn't leave me frustrated and angry at wasting time. For some automated stuff (converting text to json, etc), the local models are fine.

The offline service was still working, and people were doing their job.

The online service was not working, and it was causing other people to be unable to do their job. We had 0 control over the third party.

The other thing, I make software and I basically don't touch it for a few years or ever. These third party services are always updating and breaking causing us to update as well.

IB4 let me write my own compilers so I have real control.

----

That is a little bit dismissive of me though. There are some cool features here:

I can now "entertain in my kitchen", which is definitely a normal thing that normal people do. I love getting everyone together to crowd around my refrigerator so that we can all watch Game of Thrones.

And I can use Amazon Alexa from my fridge just in case I'm not able to talk out loud to the cheap unobtrusive device that has a microphone in it specifically so that it can be placed in any room of the house. So having that option is good.

And perhaps the biggest deal of all, I can finally "shop from home." That was a huge problem for me before, I kept thinking, "if only I had a better refrigerator I could finally buy things on websites."

And this is a great bargain for only 3-5 thousand dollars! I can't believe I was planning to buy some crappy normal refrigerator for less than a thousand bucks and then use the extra money I saved to mount a giant flat-screen TV hooked up to a Chromecast in my kitchen. That would have been a huge mistake for me to make.

Honestly it's just the icing on the cake that I can "set as many timers as [I] want." That's a great feature for someone like me because I can't set any timers at all using my phone or a voice assistant. /s

----

Holy crud, smart-device manufacturers have become unhinged. The one feature that actually looks useful here is being able to take a picture of the inside of the fridge while you're away. That is basically the one feature that I would want from a fridge that isn't much-better handled using a phone or a tablet or a TV or a normal refrigerator button. Which, great, but the problem is that I know what the inside of my fridge looks like right now, and let me just say: if I was organized enough that a photograph of the inside of my fridge would be clear enough to tell me what food was in it, and if I was organized enough that the photo wouldn't just show 'a pile of old containers, some of them transparent and some of them not' -- I have a feeling that in that case I would no longer be the type of person that needed to take a photo of the inside of my refrigerator to know what was in it.

One of the most frustrating things with "Open"AI is you can't just use what they announce as available, you have to wait for an A/B rollout (as a paying customer!) or for it to be accessible in direct way instead of going through multiple models when you just want an image.

Yes, it is available in the API (just tested to make sure the docs weren't misleading.)

But not the features themselves, not so much.

I mostly use it for writing and debugging small Bash and Python scripts, and creating tables and figures in LaTeX.

Don't need to be a dinosaur, only need to be a person willing to increase knowledge of a topic.

Google is 'cliff-notes' -compared to reading a manual- and that is not learning, it's cheating oneself.

[1] https://platform.openai.com/playground

The most useful aspect is you can provide the system prompt, and inject ChatGPT responses.

I’ve got friends who have started an incident management company. They are awesome. It feels crass to advertise for them now, but it also feels like the best time to do it.

Check out this open source Mixture of Experts research. Could help a lot with performance of open source models.

Is there a separate status page for Azure OpenAI service availability / issues?

https://status.openai.com/

Atlassian? What?