I have written myself a web fronetend using langchain that uses a json based
agent to do advanced tool use (websearch, calculations, document
summarisation/information extraction, etc) and chain of thought reasoning. At
the moment im using openai gpt-4-turbo for the main agent and gpt-3.5-turbo for
...
The only really capable models are from OpenAI, Anthropic, and Meta. They all have guard rails of some kind and you need to use a supervisor agent to decide if the output is as you wanted if you can't use something more basic like a JSON parser. Google Bard/Gemini output is not that high quality IMO.
I have a guardrails ensuring proper json output that works 99% of the time except chatgpt has been so lobotomised that it ignores that for the I can't answer that default responces. What about mistral models I've played with then but haven't found any services that let me just point at an api and fire.