Reading guest reviews with AI: from hundreds of comments to a short fix list

AI review analysis for hotels means feeding a defined set of guest reviews and survey comments to a language model and asking it to return themes with counts and verbatim quotes. It sorts text fast, but a plain summary favors frequent complaints over severe ones, so serious safety, hygiene, billing and discrimination cases need a separate pass.

A season of reviews sits in an export file and the general manager has one hour. The summary an AI writes in that hour will be clean, readable and quietly wrong in one specific way: it will talk about the complaints that came up most, not the ones that matter most. This guide shows how to run the analysis so the output is checkable, how to protect the rare serious complaints from being averaged away, and how to turn the result into a one-page fix list with owners and dates.

What does AI review analysis actually do, and where does it fail?#

It reads a defined body of review text and returns themes, counts and quotes, which is sorting work at a scale no team does by hand. It does not judge severity, and a plain summary will hide a rare serious complaint behind a common minor one.

The useful part is the structure, not the prose. A language model reads text and produces a shaped output: theme, count, quote, source, date. Prose you cannot check; a table you can. That distinction decides everything else in this guide.

What it does well: grouping hundreds of comments into a dozen themes, recognizing that "no hot water", "shower was cold" and "sıcak su yok" are the same complaint, pulling representative quotes, and surfacing a theme nobody thought to look for. That last one matters. A human reading with a checklist finds what is on the checklist; a model reading everything can return a theme you never asked about.

What it does badly: severity. Summaries compress toward the frequent, so one report of a dirty bathroom at check-in can be absorbed into a line that says housekeeping was mentioned many times. The count is correct and the conclusion is misleading.

It also has no access to your operation. It cannot know the lift was out for two days, so it cannot connect a cluster of comments to a cause. That link is your job. And it will produce confident wrong counts: every number it returns must be spot-checked against the source file before anyone acts on it.

The trade-off is real. A loose prompt gives a smoother, more readable summary; a strict output format gives a checkable one. Choose checkable. For more everyday uses of these tools across departments, see AI tools for hotel teams: safe everyday uses, department by department.

How do you prepare a season of reviews for analysis?#

Choose one defined period and one defined set of sources, remove anything that identifies a guest, and keep the source and date attached to every comment.

Write down the set: which review platforms, the in-house survey, the guest app, the front-desk complaint log. Next month's run must use the same list, or the comparison is meaningless. If you add a source mid-year, note the month it joined, because a rising count can simply mean a new pipe of comments.

Take one season, or one month, at a time. Mixing years hides the effect of a renovation, a new chef or a change of management. A theme that appears in 2023 and 2025 but not 2024 is a different story from one that runs straight through.

Anonymize before the text leaves your hands: guest names, room numbers, phone numbers, email addresses, booking references and staff names come out. The complaint itself stays. Treat reviews and survey comments as personal data under KVKK (the personal data protection law) and remove identifiers before the text goes to an outside tool; confirm your own case with a legal adviser.

Keep the metadata: source, date, language, and the rating if there is one. Without a date you cannot tell whether a theme is current or three seasons old. Keep the original language next to any translation, because a translated quote loses the exact words a department needs to see, and translation quietly softens insults and sarcasm. Remove duplicates and copy-paste reviews, which inflate a count without adding a complaint.

Store the prepared file with the date in its name. It is the baseline every later month is measured against. If your wider data needs attention first, start with hotel data quality before AI: what to clean up and why.

How do you ask for themes with counts and quotes instead of adjectives?#

Ask for a fixed output shape: theme, number of comments, one verbatim quote per theme with its source and date, and the department that owns it. Adjectives are not checkable; counts and quotes are.

State the input and the output columns in the request. If the model invents its own layout, you cannot compare this month with last month. Forbid interpretation inside the theme name: ask for "check-in wait", not "poor arrival experience". A named problem can be assigned to a person; a feeling cannot.

Require quotes exactly as written, with source and date, and no paraphrase. A paraphrase is the model's opinion wearing a guest's clothes. Ask for a second list of themes with only one or two mentions, kept separate instead of merged into a bigger theme, because the low-count list is where the odd and specific complaints sit.

Ask what it could not classify and what it may have missed. The unclassified pile is where serious complaints that do not fit a standard theme usually end up. Then verify: spot-check three themes by searching the source file for the quoted words and counting the mentions yourself. If one count is wrong, treat the whole output as unverified. Run the same request twice on the same file; if the themes move substantially between runs, the request is too loose to build a fix list on.

The output should look like this, for example:

These rows are illustrative, not real. The method for writing such requests in the first place is covered in writing prompts for hotel work: a simple method with examples.

Why do rare but serious complaints vanish from a summary?#

Because summarization compresses by frequency: one comment among hundreds is noise to the model and a safety or legal event to the hotel. The fix is a second, separate pass with fixed categories that is read in full, never summarized.

Fix the categories in advance: safety (injury, fire, lift, pool, balcony, gas), hygiene (bed bugs, mold, food poisoning), billing (unexplained charge, deposit, invoice), discrimination and harassment, accessibility, and data or privacy. These categories do not change month to month, and they are never averaged into the theme list.

Run this pass as a filter, not a summary: a keyword and phrase sweep in every language present, plus a full read of any comment with a low rating even if it is short. Serious complaints are often short. "They charged me twice and nobody fixed it" is five words and a billing case.

Read every hit yourself. The pass produces a list to read, not a verdict, and a person decides what happened. One confirmed case is enough to act. These categories are never turned into a percentage and never ranked against the theme list, because even a single confirmed case of food poisoning is not a small number.

Route each hit the same day to the person who can act, and record what was done. An unrecorded escalation is an escalation that did not happen. If a guest describes a possible injury, crime or discrimination, escalate to management and take advice; an AI tool does not decide those cases. For the face-to-face side of the same problem, see hotel guest complaints at the front desk: a recovery routine.

How do you turn themes into a fix list with owners and dates?#

Cap the list at five to seven items and give each one an owner, one observable action, a date and a way to check whether it worked.

Sort every theme into three buckets: fix now (a process or a piece of equipment), investigate (the cause is unclear), and accept (a design limit you will not change, and you say so openly). The accept bucket is not failure. A 1970s building will always be described as dated, and pretending otherwise wastes an action slot.

One named person per item, not a department. A department cannot be asked on Friday whether it is done. The action must be observable: "a second room check before key handover", not "improve housekeeping". If you cannot see it happening, it is not an action.

Set the date short enough to test inside the same season, otherwise the next review run cannot tell you anything. Define the check in advance: the count for that theme in next month's run, not the overall review score, which moves for reasons you do not control. Publish the list where the teams actually see it; a fix list in a manager's notebook changes nothing.

Log the items in the same place as guest requests, so the two lists do not drift into two different versions of the truth. The request-logging routine in hotel guest request management: log, assign and close each one works for fix-list items unchanged.

For example:

Again, illustrative only. The structure is the point: evidence, a name, something observable, a date.

How do you repeat the analysis monthly and compare results?#

Run the same prepared input, the same categories and the same questions every month, and compare counts per theme rather than the overall review score.

Keep the request text and the category list in one document with a version date. Changing the wording changes the numbers, and you will not remember which change did it. Compare like with like: same sources, same period length, same handling of languages and duplicates.

Respect small bases. Three mentions this month against one last month is not a trend, and the report should say so in the same sentence. Watch for definition drift: if "check-in wait" starts absorbing "lobby crowding", the count moves for the wrong reason and a working fix looks like a failure. When a theme's boundary changes, split the counts and say so in the report.

Keep every monthly output file. A year of them shows whether a fix held or the complaint simply changed shape. Feed the themes into two places: the public review replies and the operations meeting, so the same complaint is answered outside and fixed inside. The public side has its own craft, covered in how to reply to hotel reviews: example replies by complaint type.

What to do next#

Take last season's export and prepare it once with the checklist above: defined period, written source list, identifiers removed, dates kept. Run the two passes, the theme table and the serious-complaint filter, and put five items on one page with a name and a date beside each. The sorting is the easy part; the value is in the short list that survives the sorting, and in running the same list again next month to see whether it worked.