Evaluation of the Muse AI agent — we used Muse to evaluate Muse.
A demo of Meta's Muse model doing real travel work: planning a Kyoto + Tokyo/Hakone family trip.
This demo was created from this text:
We work in travel marketing B2B2C niche, using mostly agent portals (for now) to bridge supplier content to agents and consumers. Currently mostly legacy code and websites. This project is a quick deep dive to understand the differences and intersections between:
- Seasoned travel agent — decades of experience, deep industry and destination knowledge: tips, tricks, things to see and things to avoid. People who actually use great travel agents rave about them afterward — pitfalls avoided, better experience, often saving money.
- Beginner travel agent — someone new, or under a decade in the industry. Where is the gap between their business and a seasoned veteran's? How can we, in the B2B2C niche, bring more value to the average agent — taking them from beginner to offering customers the ultimate curated travel plans? That means identifying exactly what value a travel agent provides, why a customer should prefer one, the best marketing strategies, and the best ways to add value for travel suppliers so their products get visible and highlighted to agents and agent customers.
- LLM-driven "Travel Agent" — the models and their harness intelligence curve are pointing up, so we ran a direct test of what an LLM agent can do today: a quality travel-agent experience delivered from an LLM. Once complete, we compare that system's strengths and weaknesses against a quality experience with a human travel agent.
- Extrapolation — from this data, the strengths of in-person contact versus artificial intelligence: reasons plus potential tools we can offer agents and suppliers to create a strong connection to their customers — trust, friendship, and knowledge for their next adventures.
Outputs: markdown for each of the areas above — clear, concise, easy for non-technical people to read. Solutions, not problems: what is available today, and anticipate tomorrow, so the solutions we build today are not obsolete in a year or a month. Plus a very simple LLM-based virtual travel agent demo — a Python app with an LLM call, a clean professional UI showing a travel portal an agent might use (prefilled with our Kyoto + Tokyo/Hakone test trip), trip results in a clean scannable format with follow-up questions and share/book actions. And as a summary, an HTML page of what we learned: B2B2C solutions for today, tomorrow, and the foreseeable future.