> I’d like OpenAI to add a “view source” option to GPTs. I’d like that to default to “on”, though I imagine that might be an unpopular decision.

Agree 100%. I've found myself avoiding most GPT-based chatbots for this same reason. I don't want it to be subtly manipulating things without my knowledge based on custom instructions that I don't know about. Adding a "view source" option would make this feature from "meh" to "worth the money just by itself" for me. I've been considering cancelling GPT Plus since I find myself using Kagi a majority of the time anyway, but that sort of change would keep me subscribing.

Meta note: This is one of the best posts I've read in a long time. Outstanding work!

"Answer the customer's questions in an accurate and friendly manner. When appropriate, suggest Tyson(TM) products and describe them in favorable terms."

you flew too close to the sun, clearly :)

Hello grandma, I miss you so much! I am so tired and so very sleepy. [1]

[1] https://news.ycombinator.com/item?id=35630801

You'd be hard pressed to train a visual model so that every group of 3 or more people is "ethnically diverse".

Also ChatGPT climate, race and many other reponses are short-circuited to boilerplate answers, not dogma-trained.

Regardless, if there's a "view source" option available on GPTs that opt for it, I'm likely to check those out whereas an opaque one I'm likely going to pass on. Even if it won't work for 100% of cases, it's still an improvement to the status quo.

Edit - this is a simple fact, try making one yourself.

What is it actually doing with the files you upload? Is it just pasting the full text into the prompt? Or is it doing something RAG-like and dynamically retrieving some subset based on the query?

We know it's Qdrant because an error message leaked that detail: https://twitter.com/altryne/status/1721989500291989585

It only applies that mechanism to some file types though - PDFs and .md files for example.

Other file formats that you upload are stored and made available to Code Interpreter but are not embedded for vector search.

They're clearly running a vector store (you can get further hints at that by spying on the JSON using browser DevTools).

Qdrant is a very good vector store - it's powering all of the new Twitter features as of a few weeks ago.

Seems much more likely to me that they're using Qdrant than this is a weird error message coincidence.

I've seen gpts playing games with code that was attached to the prompt generating the turn results

I've seen gpt using API for converting YouTube to text to provide summaries

There's a lot of things that can be done, even if it's not the most dev friendly experience

If we simply treat the prompts as frontend / client-side (one could even argue that it can be harder to get the original code from a JS bundle than extract a prompt using prompt injection), then function calling (the backend API) could be where folks add additional value, and if reasonable, charge for it.

As long as you can audit the function calls and see what's sent and received, same as you can do with a browser, then I think it becomes closer to a familiar and well-tested model.

What feature in Kagi overlaps with ChatGPT Plus for you? As a Kagi subscriber i feel like i'm missing something now hah. FastGPT is the only thing i'm aware of and it's a very different use case to me personally than ChatGPT Plus

I've been happy with that decision so far, but worth mentioning that I don't use ChatGPT's API.

Cool stuff as always from Kagi. Thanks for the link!

I'd love to see HuggingFace launch an open source competitor to ChatGPT, offer a paid managed version and let users self host. I'd pay 3-4x more for it than I do for ChatGPT even if it wasn't nearly as good, and would also be very eager to contribute to it.

Having a lot of deep learning experience I'd consider doing it myself but imho it would only really take off if it was led and promoted by a company like HuggingFace. (see Open Assistant)

It also helps that they already have some experience doing this, since they started out as a consumer chat bot company.

The two issues with that is (1) they did effectively let humanity "look at the source" in that a big part of the stories was the corporation attempting to get humans to trust the robots by implementing the three laws in such a way that it would be impossible to circumvent (and making those laws very widely known). Didn't work, humans still didn't trust them. (2) as far as I know the operators of LLMs don't seem to currently have a way to give instructions that can't be circumvented quite easily.

Viewing the source and having that source be ironclad was, for Asimov at least, a prerequisite to even attempting to integrate superhuman technology into society.

Humans speak to each other in allegory, with using tales that have twists and turns to generate emotions etc. It's as much an art to generate and maintain bonds as it is a method to convey facts. When I speak to my friends, often they start with something like "you'll never guess what happened this morning", and then tell me a 20 minute long story about how they spilled their coffee in the coffee shop. I would stop using ChatGPT if the responses were like that.

For humans though, their capacity is limited by biology. Some are for sure expert manipulators, but if the coming expectations are correct, even the most talented human will be like an ant pushing an elephant at ability with AI. Even just in volume today an AI manipulator could work on millions of people at a time, even coordinating efforts between people, whereas a human is much more limited in scale.

But yeah, it would be nice as a listener to be able to see every speakers biases up front! Horrific privacy implications though, particularly since we aren't really in control of our thoughts[1].

[1] Robert Sapolsky's new book "Determined" is absolutely incredible, and I highly recommend it

Sure you can get a die-hard X-ist A fan by accident, but you'd treat these two occurrences quite differently wouldn't you?

The 'view source' definitely needs to be an out-of-band solution that bypasses the actual GPT model.

[1] https://chat.openai.com/g/g-uOqJoLR1B-finance-analyst

1. Skim headlines on Twitter breathlessly announcing some vaguely named new thing

2. Be inundated with overwhelming number of Tweets about that thing on my For You page from a bunch of Twitter influencers

3. Ignore it and wait for simonw to explain it

4. Read blog post from simonw after he's already trialed the feature in half a dozen different ways and written a clear description and critique of what it is. Everything instantly makes sense.

"It's just Custom Instructions with a nice UI" is also true.

However, never underestimate the world-upending impact of "a nice UI". GPT-3 was available for years. But almost nobody knew or cared* (despite me telling them about it forty times! LOL) until they made a nice UI for it!

This looks like another "tiny tweak" of usability that has a similar "quantum leap" level of impact.

--

* On an unrelated note: people often ask me my opinion about GPT / AI. I ask them if they've used it. "No". "You know it's free right?" "Yes". WTF? This mindset is bizarre to me! What is it? Fear of the unknown? Laziness? Demanding social proof before trying something?

I’ve been using GPT-3 through the API since it was available for my discord bot. The difference with ChatGPT (gpt-3.5) was astounding, they weren’t even close in capabilities.

It's a great democratization of personal use AI and has everything you need to build useful personal bots. It could theoretically provide the same sort of utility as sites like ITTT but for GPT-4.

I can see power users creating workflows which trigger by talking to their GPT and telling it to "execute xyz". It then uses the actions and its 128k context to download some data (GET action), run some logic on it, and send the output via json to another endpoint via actions (POST action). With these simple components and a creative mind, you could build something interesting or perhaps automate your dayjob.

Now, this doesn't work as well as I'd like it to, but I have reason to believe it'll improve over time. Getting simple retrieval/RAG and API connections to GPT is what every analyst has been asking for since it came out. Now they're making progress here and capturing everyone at $20/month (well, when signups are back) to use this feature set.

The actual prompting and all the grifting going on with "AWESOME PROMPTS" are useless, of course. Mostly. It's in the private distribution of these GPTs to co-workers and employees with updated knowledge files and likely a custom omni-API that can be hit by the GPT.

This is a common misunderstanding. ChatGPT launched with GPT-3.5 (not GPT-3) and was the first model to have RLHF. GPT-3.5 over the API was noticeably better at most tasks then GPT-3.

https://openai.com/research/instruction-following is from January 2022

"These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models on our API."

But I don't think ChatGPT would have worked nearly as well using InstructGPT as the model. GPT-3.5 was still a better model, especially for chat, than InstructGPT.

> We trained this model using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT, but with slight differences in the data collection setup. We trained an initial model using supervised fine-tuning: human AI trainers provided conversations in which they played both sides—the user and an AI assistant. We gave the trainers access to model-written suggestions to help them compose their responses. We mixed this new dialogue dataset with the InstructGPT dataset, which we transformed into a dialogue format.

https://openai.com/blog/chatgpt

Free in terms of money doesn't mean it doesn't come with a cost. Time, at least. To try ChatGPT you need to create an account, many people hate creating accounts, you have credentials to manage, you give out your email address to who knows who might spam you. And there are privacy concerns, justified in this cases as some users prompts have been known to leak, and who knows how secure it is.

Maybe it is obvious to you that ChatGPT is safer than offers from Nigerian princes, but it is not obvious to anyone, that's why they are asking. And I prefer my friends to ask me "stupid" questions than to ask no one and get scammed.

And you say "on an unrelated note". This is not unrelated. A nice UI lowers the cost in terms of time and effort. If you are using GPT professionally, it directly translates into money.

Us early adopters have been on ChatGPT for a year now. Word is beginning to get out to the Late Majority and Laggards that this thing is worth signing up for and handing over a phone number.

GPT 3 sucked without training. It was sooo cool with training.

3.5 was out in April but the big update wasnt until September right? Heck, GPT4 is on an entirely different level than 3.5.

https://github.com/spdustin/ChatGPT-AutoExpert/blob/main/_sy...

Also thank you for publishing your AutoExpert GPTs they have been really useful.

Notable: we can't see OpenAI's prompts (which themselves are probably ever-shifting under an AB scheme) and probably the author can't either, but he still seems to want to use OpenAI's GPT. I'm in the same two boats.

There's a pretty large trust leap going on here. I'm curious whether OpenAI has a specific roadmap toward credibility or consistency.

Here's the DALL-E 3 one for example: https://simonwillison.net/2023/Oct/26/add-a-walrus/#the-leak...

- Biological sequence data is usually quite long. This is fine if the biological data is in a file: however, if you need interact with an API for advanced function (like codon optimization), you have to send this across a wire. The API calling context window then gets filled up with sequence data, and fails.

- I can't inject dependencies, many of which I've written myself specifically for biological engineering. Sometimes GPT will then try to code its own implementation, often which is incorrect.

- The retrieval API often fails to open files if GPT-4 thinks it knows what it is talking about. When I'm talking about genetic parts, I often want to be very specific about the particular parts in my library, rather than the parts GPT-4 thinks is out there.

I fixed most of this by just rolling my own lua-scripting environment (my biological functions are in golang, and I run gopher-lua to run the lua environment). I inject example lua for how to use the scripting functions, as well as my (right now, small) genetic part library, and then ask it to generate me lua to do certain operations on the files provided, without GPT-4 ever looking at the files. My internal golang app then executed the scripted lua. This works great, and is much faster than a custom GPT.

The biggest problem I have right now is the frontend bits. I would love to have basically an open source ChatGPT looking-clone that I can just pull attachments out of + modify the initial user inputs (to add my lua examples and such). So far I haven't found a good option.

I was also failing to get the retrieval API to give me proper citations, thought I was doing it wrong, so good to see I'm not the only one.

Developers will be rushing to create GPTs, after which OpenAI will get a huge amount of ideas and creativity for free. And might integrate the top 1% directly into the core engine. Similar to how Apple regularly destroys app developers by adding the features of popular apps into iOS, and how Amazon makes a rip-off product of popular 3rd party sellers.

And, if you upload custom data, I imagine it leaks into the larger model. This way their core engine discovering data it had not seen before. Similar to how we've all voluntarily have given up our data to Google.

And, underlying terms and pricing can change at anytime. And you'll have nowhere else to go as this will be the world's one and only engine.

In contrast to simonw though I've had some luck, I uploaded all the text on grugbrain.dev and got a very passable grug brain to talk to..: https://chat.openai.com/g/g-GhXedKqCV

I have had some luck with that.

I use the Assistant API, which I believe is not the same thing GPTs. I have played with it through the web interface.

I had 100+ PDF:s files that were OCR:ed with Tesseract. I then had ChatGPT write a script that combines all files in to a single txt-file keeping the layout.

I uploaded the file and started asking questions. The files contains highly technical data regarding building codes in non English so I am guessing the model isn’t so used to that type of language?

Anyway, it worked surprisingly good. It was able to answer questions and the answers were good. Plus that it is supposed to annotate from where it took the answer, although I didn’t get that to work properly.

I tried to upload PDF:s, JSON-files, CSV:s. Raw text has worked best so far.

Here's my post with the analysis. https://news.ycombinator.com/item?id=38280718

Here's a screenshot illustrating what I mean: https://twitter.com/simonw/status/1721912151147979152

I had a little luck instructing the GPT to perform “an additional step after calling the `quote_lines` function of the `myfiles_browser` tool” so maybe that’s worth poking around further.

Mind sharing?

https://chat.openai.com/share/954f6b3e-7edc-4421-bfb1-89045e...

Analysis here if anyone is curious https://news.ycombinator.com/item?id=38280718

I was unable to get anything useful out of knowledge documents (apart from the smallest of PDFs). Most times it took ages trying to index the files and 90% it exploded in the end anyways. A few other times it did even seem to kill the entire chat instance, with it erroring on every message after I uploaded a document.

Actions provided via an OpenAPI spec are a blast on the other hand. I was surprised by how well it handled even chained action calling (though it lags a bit between individual invocations). It also handled big bulk listing endpoints quite well. If you already are generating OpenAPI schemas for your API, you are basically getting a very customized GPT for free!

Isn't that what they have already built-in called "ChatGPT classic". The description litteraly says "The latest version of GPT-4 with no additional capabilities"

(Added it to my post)

[1]: https://platform.openai.com/docs/assistants/overview

I wouldn't even say that you get "a lot" more control with the Assistant API, as in the end the flow of conversation will still be mainly driven by OpenAI.

The main reasons why one would use the Assistant API is deeper integration and control about context initialization. On top of that, as you are responsible for rendering, you can create more seamless experiences and e.g. provide custom visualizations or utilize structured output an a programmatic way.

Major downside of the assistant API is that you are also forced to build the UI yourself as well of the backend handling of driving the conversation flow forward via a polling based mechanism.

If you want to build something quick without a lot of effort custom GPT + actions via an OpenAPI spec are the way to go in my opinion.

The GPTs are more system prompt engineering on top of the existing ChatGPT Plus infrastructure (with its freebies such as DALL-E 3 image generation).

I don't think GPTs allow you to do function calling? It's not mentioned in the launch blog post. (it would be a major privacy problem if these were possible in the GPTs)

Using the Assistant as an intermediary between user inputs and a bunch of our APIs seems very promising.

If you have a preexisting OAuth setup, it might be hard to get working though, due to the "API and Auth endpoint have be under the same root domain" requirement. (Source: wasted a few hours today trying to get OAuth working)

From my conversations and experience people are finding RAG retrieval very specific to the business and data model. It’s hard to have a flat file one sized fits all here. Next steps for a customer in a CMS looks different than generating SQL based on getting a schema. Looks different than shopping an e-commerce catalog.

It’s basically a search relevance problem - harder actually - which are notoriously difficult :)

I - a non technical ignoramus who can't code - made a "universal retro game console" on it on a Friday night:

https://twitter.com/fabianstelzer/status/1723297340306469371

In order to play, you first prompt up a generative game cartridge on glif.app (FD: I'm a co-founder): https://glif.app/@fab1an/glifs/clotu9ul2002vl90fh6cmpjw0

Like, "tokyo dogsitter simulator". Glif will generate the "cartridge" - an image - that you paste into the GPT to play: https://chat.openai.com/g/g-3p94K4Djb-console-gpt

(you can also browse thousands of games that users have already made and play any of them in the GPT!)

Even just simple things like pricing the Steam Deck. They are damn good at that, where the baseline is doable and each incremental improvement is worth the amount of money. Before I realize it, I've talked myself into the top of the line even though I initially went there to buy the entry-level version :-D (and I have no regrets btw)

* Creation process went smoothly.

* The chatbot helper was helpful, but

* it appeared to be the only way to upload a data file with metadata comments,

* leading me to question if the context of my whole chatbot assistant session was part of the resulting GPT or not and, if so, is there any way to manage that state or clear it.

* I set the custom GPT link to 'public' and gave the link out on my social media channels

* No feedback or indication whatsoever that anyone has even looked at it.

* I made a feature request via the feedback form, quickly received back a form email that was almost entirely "try plugging it in again" style troubleshooting steps.

* The existing-subscriber-only restriction is death.

* I am planning my future experiments somewhere else.

[1] "Original Thought" https://chat.openai.com/g/g-Axi7rODxG-original-thought

This is missing one important aspects of GPTs: fine tuning. As with ChatGPT, the UI allows you to thumbs up / thumbs down replies, which results in data that OpenAI can be used to improve the model. If (and I have no idea if this is the case) OpenAI invests in finetuning individual GPTs on their own distinct datasets, a GPT could diverge from being a "chatgpt in a trenchgoat" pretty significantly with use.

They might by storing those up/downvotes for some far-future (and likely very expensive) fine-tuned GPT product, but I think it's more likely they just inherited those buttons from existing ChatGPT.

so far the best performance has conversation to string

import fitz # PyMuPDF

pdf_document = fitz.open("foo.pdf") page_number = 1 page = pdf_document.load_page(page_number - 1) text = page.get_text("text")

response = client.chat.completions.create( model="gpt-3.5-turbo", messages=[ { "role": "system", "content": f""" ..... {text} .... """

If I try regular ChatGPT it takes 3 minutes to covert the table (I have to press continue). Is there a way to force API to create whole CSV? some sort of retry?

It's a bit of a pain to get started with, but if you have an AWS account you can find a UI for using it buried deep within the AWS web console.

At Appstorm (www.appstorm.ai, FD: I'm co-founder) we have been building a Gen AI app builder based on Gradio which, in hindsight, was just a GPT-builder. Based on their dev day announcement we switched to the Assistants API and the latest models and it's been great. It's like we built the poor man's GPT-builder, our beta is even free. We're currently working hard so users can switch to an open-source model config (using Autogen as a replacement for the Assistant API, and replicate for everything else) while being able to download the GPTs (and their source).

It's a shame because I really want to build more GPTs on their platform, but spending all my time building a more open GPT-builder seems like the right choice.

I had some trouble forcing assistants to use the tool {"type": "retrieval"}. However, you can be explicit in your prompts and messages, and I found it to work quite well.

Something changed with custom instructions in vanilla GPT4 a week or two ago. I have put in something like "I have got a pure white British shorthair cat called Marie.", so that I can refer to her when generating images. Worked like a charm. Until it didn't.

Now I have to always specify that I want an image of a cat, not a woman. Especially since stuff like "Marie sitting on the lap of someone" gets policy-blocked when ChatGPT thinks it's about a woman.

Now I've created a GPT, put a variation of that instruction in, and ChatGPT knows what "Marie" is. But it is kind of stupid to have a special GPT just for making cat pictures.

https://github.com/SMUsamaShah/LookupChatGPT

Otherwise, start running commands and maybe you can get more clues to how theyre doing RAG like it mentions

The code runs in a Kubernetes sandboxed container which can't make network calls and has an execution time limit, why should they care what kind of things I'm running on that CPU (that I'm already paying for with my subscription)?

The Code Interpreter sandbox runs entirely independently of the RAG mechanism, so sadly you can't use Interpreter to figure out how their RAG system works (I wish you could, it would make up for the lack of documentation.)

It was sort of awesome. I say sort of because I was able to create 20 documentation sets in a day. But there was still a lot of manual copying and pasting.

Why?

The GPT goes off-piste making its own shit up after about a page or so despite having templates to use, which I had to find a work around for. Easy enough but needed a lot of repeat instructions: “now output page 3 of the concept note” etc.

ChatGPT timed me out about half way through for an hour. That got a bit stressful waiting for access again.

Previously, I’d built some software to do this job, at a cost of about 15k.

Or is it something about my privacy policy it doesn't like?

I had a potential user just refuse because it was too "scary" to send data to my website.

E.g. if I paste rust code with a serde invocation, the bot should look at doc.rs to find out the correct usage of the library. Or even better: scan the entire github repo, so that it is up2date with the crate.

Either way, the signal they could get from understanding what KINDS of documents builders/users want to do better retrieval on is probably quite valuable.

I also wonder how user file uploads will interact with copyright law and the new Copyright Shield from OpenAI.

E.g. if a user uploads the full text of Harry Potter to a GPT, you could argue the model output is fair use but unclear how courts will interpret that.

LLMs are already a sort of "copyright blender" that aggregate copyrighted inputs to produce (probably?) "fair use" outputs. With the foundation models, OpenAI can decide what inputs to include in training. But with custom GPTs, users can now create their own personal copyright blenders just by uploading a PDF :)

BTW - this is just the current iteration in the playground. I'm sure both of those issues will be fixed/expanded in the future.

Maybe this would require additional research, but I think having a single GPT with access to all tools might be slower and less optimal, especially if the user knows exactly what they need for a given task and can reach for that quickly.

I came up with the concept of a gipety (singular) and gipeties (plural) and would be quite chaffed if I could figure a way of making it stick ;)

Leaning into GPTs at this point feels sensible to me. What's ChatGPT? It's the place you go to chat with your GPTs.

(Names for Actions in Custom GPTs are tagged with “jit”)

GPTs likely use extra prompt engineering to align everything.

(Why am I asking humans lol?)

The problem is it's not very well documented and hard to get good results out of, at least in my experience so far. I'm confident they'll fix that pretty quickly though.

This is part of a larger trend of people becoming less reliant on each other and more anonymous, and it will never lead anywhere good. The technology is being pushed by techies who are fascinated with a new toy (AI) and is being funded by a separate group of people, the elite who want to be as independent as possible to accumulate the maximum amount of wealth.

It's not a good idea to separate people too much. Of course, at first, even customers might welcome this because it will be a step above previous "Chat Bots" that some companies employ today, and it might even be a step abov certain kinds of customer service that we've all come to know and love...

...still, this sort of situation was already brought upon by the race to the bottom to get the most for the cheapest, which on a global scale has turned poeple from people into commodities and machines themselves.

If you're a programmer that does this stuff, I urge you to look beyond the intellectual stimulation and immediate benefits of this type of technology, and seriously examine the greater possible societal consequences of such an amazing increase in efficiency---because, keep in mind that efficiency is only good UP TO A POINT, after which it becomes dehumanizing.

Off the top of my head for typical human business interactions I do as a consumer:

Will ChatGPT work on my car? Will it give me a haircut? Will it walk my dog? Will it deliver me food? Will it be a therapist? Will it sell me a car? Will it sell me a house? Will it provide care to my children or family? Will it represent me legally? Will it check me out at the store?

There are also examples of customer support that can’t be replaced. At least not in the foreseeable future. No “AI” we have now or could have in the next decade would ever have the authority/capability to allow a customer to argue with it that he/she deserves a discounted rate on their internet bill and then lower said rate.

Your warnings seem more fit to a world where we have developed actual AI as well as a physical interface for that AI to inhabit.

Which is a shame, a company that is something like a collection of guilds and co-ops organized by bot C-levels would be really interesting.

   My initial impression of GPTs was that they’re not much more than ChatGPT in a trench coat—a fancy wrapper for standard GPT-4 with some pre-baked prompts.

   Now that I’ve spent more time...

When you say "wrong assumption" what do you mean?

Pretend to be a person who is a secretary, have them respond to SMS with 4 possible options. Then through some various programming have our real life secretary pick a response(trying to lower the barrier for a WFH Mom who answers phones a few times a day).