Coming SOON!?
Sorry openAi, I want to play with it :)
Coming NOW.
DELVE IN
Also of note, the codename for this project appears to be "Sidekick". In case you were wondering where the app is headed
Also the default hotkey for the bar conflicts with Arc browser mini arc hotkey
RIP
There can only be ONE
ngl Im kinda hitting UI bar overwhelm
cmd+space for raycast
option+space for chatGPT
now I'm trying cmd+y for mini arc since its near cmd+t for arc's new tab & command bar
cmd+K in tons of apps, notably cursor
• • •
Missing some Tweet in this thread? You can try to
force a refresh
-a lazy coder
Peculiar that both GPT and gemini are running into this.
Gemini seems worse. Particularly because it refuses to fix it and fill in
-double check response, modify response, see drafts, and try again without extensions, are very intriguing and nice touches
-still randomly refuses to do simple tasks...
Absolutely infuriating experience. It can draw a goblin, but a goblin with a sword is impossible apparently. Can't draw images, even though it did that 2 minutes ago
-really really bad at conversation turns.
often ignores instructions, repeats patterns, refuses to do things. Consistently takes things in super broad contexts and basically forgets what we were talking about. After writing incomplete code, I asked for "all code" and it gave me a disclaimer that it cant write all code in the universe...
-extremely disclaimer heavy. more annoying that GPT4 by far. But probably a good idea given their huge customer base.
-being able to access email and docs is nice, but kinda broken because it assumes I have 1 google account. But I have like 5 google emails, and I'm not signing up both my work and personal for 20/mo
-editing your prompt destroys chat history. No forking 2/3 chats to arrow through.
-generate more images button doesnt work half the time
-hate that it pulls in my location. with no option to turn off
-fast, but the load anim is weird. I don't like that it scrolls my message to the top after entering. (tbh I hate the aesthetics of Google's material design in general.)
-left bar makes no sense. Why is there a drop down button when theres plenty of room to show my history?
-image quality is hit or miss. Far behind midjourney, dalle & playground. Especially faces. Gets progressively worse as you ask for changes.
---
Overall:
Decent release,
Feels far behind GPT-4 imo
Uninspired. Nothing inherently new. Playing catch up.
Except perhaps the double check search? which feels like table stakes for GOOGLE.
Gonna keep playing with it, but as it stands after 30 mins of testing, I likely won't sign up for the subscription after the free trial is up.
Might make more sense for people with lots of google docs?
By far my biggest issue is it randomly decides it can't help with things sometimes, when it clearly can. I would much rather it try and fail, then just say "I cant do that"
Trying to get it to write code, and not give commentary is basically impossible
Here's another good review. I agree with most of it
Definitely a different tone. Less mechanical, and more wordy. Might need to change prompting style
Though I'm very skeptical of using benchmarks to place Gemini in GPT-4 tier
They destroy prompt performance and remove's the ai's ability to coherently follow the system prompt
Why?
When you insert docs, the system copies snippets of your docs into the system prompt & context window
So 10 docs. Each around 4000 letters. FLOODS the entire context window
Which means your ai is now too stupid to remember your last message. Less memory than a goldfish
And now the system prompt is 10x as long, so your instructions get ignored
Workarounds & fixes:
Pink knowledge docs are available in the context and via RAG
Blue are only via code interpreter, and work MUCH MUCH better, especially as things scale up
-Use .md files, which are not compatible with knowledge retrieval, and open them by writing code, with code interpreter. This keeps them out of the system prompt context window, only fetching when needed. Dramatically improving overall performance
(Downside here is doc search becomes way worse. Better when you want the full doc RAG'ed in. Not good for summarizing.)
-DO NOT USE ALL 20 FILES, or if you do. DO NOT MAKE THEM LARGE. I had 10 files at ~4000 characters which destroyed all multi step reasoning. It would literally forget and repeat the first message in the conversation over and over again. Swapping from .txt to .md instantly fixed this
-Only include docs which include working system prompts for the ai. Do not simply chuck a doc in here and expect it work
I think "documents" end up all 3 of 1. Searchable vector db via "Searching Knowledge" 2. Inside the system prompt as part of the tool "instructions" 3. In the sandbox data mnt, available to code interpreter
Introducing
Evolution Chamber
A custom GPT to mutate your custom GPTs
-Build GPT actions easily
-OpenAPI schema generator, specific for GPTs quirks
-Validation tools with better error messages
-A template to build your own backend server to handle actions