This is more of a VAD/turn detection issue. It's gotten a lot better over the last few years, but it's a hard problem. The extra ~100ms of latency makes a huge difference otherwise, especially when you have use cases that require tool calling that can easily add 500ms+ of latency.
If you have tool calling complex enough that it necessitates a higher reasoning level, and you would otherwise have reasoning set to "none", this can easily come out to 500ms.
I expected the same out come you're saying here, but in my experience this hasn't been the case. I've been researching new acoustic guitars to purchase, and I've been getting an equal amount of suggestions from the major brands and the small brands.
Part of it though is I'm giving lots of context (e.g. guitar player for 10+ years, huge Opeth fan, looking for something with as close to an Ibanez style neck as possible under $1000)
I think guitars market is kind of exception because it is pretty normal for guitar players to search for "guitar like fender but cheaper". There are tons of reddit/forum discussions about this and those small brands are actually very well known in community, because majority of guitar players play on cheap instruments. Youtuber Phillip Mcknight often talked about that cheap guitars move in ridiculous volumes compared to more expensive ones like Gibson or Fender.
Had the pleasure of working with Alex while at System1. Great guy. If I remember correctly I got one tiny change merged into Waterfox that's probably since been undone in the years since :-).
It might be role-specific. I'm a solutions engineer. A large portion of my time is spent making demos for customers. LLMs have been a game-changer for me, because not only can I spit out _more_ demos, but I can handle more edge cases in demos that people run into. E.g. for example, someone wrote in asking how to use our REST API with Python.
I KNOW a common issue people run into is they forget to handle rate limits, but I also know more JavaScript than Python and have limited time, so before I'd
write:
```
# NOTE: Make sure to handle the rate limit! This is just an example. See example.com/docs/javascript/rate-limit-example for a js example doing this.
```
Unsurprisingly, more than half of customers would just ignore the comment, forget to handle the rate limit, and then write in a few months later. With Claude, I just write "Create a customer demo in Python that handles rate limits. Use example.com/docs/javascript/rate-limit-example as a reference," and it gets me 95% of the way there.
There are probably 100 other small examples like this where I had the "vibe" to know where the customer might trip over, but not the time to plug up all the little documentation example holes myself. Ideally, yes, hiring a full-time person to handle plugging up these holes would be great, but if you're resource constrained paying Anthropic for tokens is a much faster/cheaper solution in the short term.
Yup, LLMs are rocking for smaller more greenfield stuff like this. As long as you can get your results in 5-10 interactions with the bot then it works really well.
They seem to fall apart (for me, at least) when the projects get larger or have multiple people working on them.
They're also super helpful for analytics projects (I'm a data person) as generally the needed context is much smaller (and because I know exactly how to approach these problems, it's that typing the code/handling API changes takes a bunch of time).
1000%. When the sale doesn't go through, it's the salesperson's fault. When the product doesn't work, it's the "real" engineer's fault. When everything works, the client gives you a high five.
If you don't know the answer, you can ask one of the "real" engineers.
As long as you show up with a smile on your face and the demo kinda works during the call, you're 10/10.
At FAANG companies, you generally get paid at a level above your technical role; for example, if you have a mid-level engineer's coding ability but can also talk to customers, you'll generally be paid a senior engineer's salary.
Some days, I don't understand why everyone doesn't want this job. But then I'll talk to the product engineers on my team, and they'll thank me for talking to the customers so they can focus on coding. I think it's really a personality/preference thing.
Yup, it is. It's my bread and butter too. So much so I decided to just do it for myself and start my own consulting company.
Being a solutions engineer at the right companies means you get to be one of the few people with full end-to-end visibility of the entire lifecycle of both a client and the technology adoption, deployment, optimization, maintenance, etc. process. And you'll get to see it dozens or hundreds of times for a variety of clients across industries. Again though, totally depends on the company.
@cootsnuck, if I didn't really love working with the people/company I'm at now, I'd also start my own consulting company.
Once I realized you really only need 3-5 consistent customers (well, you only REALLY NEED one customer), and you can generally keep customers and employees happy by responding quickly and doing what you say you'll do (aka not taking on work you can't handle) I'm confident I could branch out on my own if I ever wanted to.
This is one of the few hills I will die on. After working on a team that used Phabricator for a few years and going back to GitHub when I joined a new company, it really does make life so much nicer to just rebase -> squash -> commit a single PR to `main`
What was stopping you from squash -> merge -> push two new changesets to `main`? Isn't your objection actually to the specifics of the workflow that was mandated by your employer as opposed to anything inherent to merge itself?
I agree with you for many use cases, but for the use case I'm focused on (Voice AI) speed is absolutely everything. Every millisecond counts for voice, and most voice use cases don't require anything close to "deep thinking. E.g., for inbound customer support use cases, we really just want the voice agent to be fast and follow the SOP.
If you have a SOP, most of the decision logic can be encoded and strictly enforced. There is zero intelligence involved in this process, it’s just if/else. The key part is understanding the customer request and mapping it to the cases encoded in the SOP - and for that part, intelligence is absolutely required or your customers will not feel „supported“ at all, but be better off with a simple form.
What do you mean by "such a system"? One that uses AI to funnel your natural language request into their system of SOP? Or one that uses SOPs to handle cases in general? SOP are great, they drastically reduce errors, since the total error is the square root of the sum of squares of random error and bias – while bias still occurs, the random error can and should be reduced by SOPs, whenever possible. The problem is that SOPs can be really bad: "Wait, I will speak to my manager" -> probably bad SOP. "Wait, I will get my manager so that you can speak to them" -> might be a better SOP, depending on the circumstances.
It never works. You always just get the digital equivalent of a runaround and there simply isn't a human in the loop to take over when the AI botches it (again). So I gave up trying, this crap should not be deployed unless it works at least as good as a person. You can't force people to put up with junk implementations of otherwise good ideas in the hope that one day you'll get it right, customer service should be a service because on the other end of the line is someone with a very high probability of being already dissatisfied with your company and/or your product. For me this is not negotiable, if my time is less valuable to you, the company, than it is to actually put someone on to help then my money will go somewhere else.
right and keep piling slop over slop, software will collapse with this mentality. And more importantly the more the code is convoluted the more even the llm will bail out and won't be able to make further adjustments because of bad code and context rot
reply