访谈结果报告生成

目的

我们已经收集很多访谈的结果，根据访谈的结果生成1份报告。

数据

访谈的基本大纲:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Meituan Takeaway User Experience Interview Outline
Interview Background: Through this interview , we hope to gain an in-depth understanding of users' experiences with Meituan Takeaway, so that we can better optimize products and services in the future, improving user satisfaction and experience.
Interview Objective: To better collect feedback on users' experiences with Meituan Takeaway, in order to optimize and improve the service in the future.
1.  Information
Could you briefly introduce yourself (e.g., age, gender,occupation, hobbies)?
How many times do you use Meituan Takeaway per week on average?
2. Usage Scenarios
What scenarios do you typically use Meituan Takeaway?
What types of products do you mainly purchase on Meituan Takeaway?
3. Satisfaction
How satisfied are you with Meituan Takeaway overall? If you were to rate it out of 10, what score would you give?
What do you consider the leading advantages of using Meituan Takeaway? Please explain briefly.
What do you consider the leading disadvantages of using Meituan Takeaway? Please explain briefly.
4. Specific Points
Order Process: Do you find Meituan Takeaway’s ordering process simple? Have you encountered any issues?
Delivery : Are you satisfied with the delivery speed and service attitude of Meituan Takeaway couriers? Please describe briefly.
Food Quality: Are you satisfied with the quality of the food you order through Meituan Takeaway?
After-Sales Service: What are your thoughts on Meituan Takeaway’s after-sales service (e.g., refunds, complaint handling, invoice, etc.)?
App Experience: Have you experienced any technical issues or operational inconveniences when using the Meituan Takeaway app?
5. User Habits and Behavior
What do you think of the overall pricing of Meituan Takeaway’s products? Does it meet your expectations?
Do you often pay attention to or use Meituan Takeaway’s promotional activities? How do these activities affect your usage frequency?
6. Suggestions and Expectations
What do you currently think are the areas that Meituan Takeaway most needs to improve?
What new functions or services would you like Meituan Takeaway to offer in the future?
7. Closing
Do you have any additional feedback or comments about your experience with Meituan Takeaway? If so, please describe briefly.

Thank you very much for your participation and communication. Your feedback is very valuable and important.

一个访谈的样本：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
{
    "user_id": "user20240703052842",
    "interview": [
        {
            "question": "Could you briefly introduce yourself, including your age, gender, occupation, and hobbies, and also share with us how many times you use Meituan Takeaway per week on average?",
            "answer": "I am 33 years old, male, a programmer, and I enjoy hiking. I use Meituan Takeaway about once a week."
        },
        {
            "question": "Can you walk me through a typical scenario where you use Meituan Takeaway, such as what time of day, what type of food you usually order, and what prompts you to open the app and place an order?",
            "answer": " I often use Meituan on weekends to order food that I can't cook myself or that is more complicated to prepare. I usually place orders through the mini program."
        },
        {
            "question": "Can you elaborate on what specific types of food you typically order on Meituan Takeaway, and are there any particular cuisines or restaurants that you prefer or frequently order from?",
            "answer": " I often order pizza and I like \"Station Pizza.\""
        },
        {
            "question": "Can you walk me through a recent experience you had with ordering pizza from \"Station Pizza\" on Meituan Takeaway, and how did the ordering process, delivery, and food quality meet your expectations?",
            "answer": " Yes, ordering through the mini program is quite convenient and fast. You can see the delivery location in real-time, and the pizza is still hot when it arrives. My favorite is durian pizza."
        },
        {
            "question": "Can you compare the food quality of \"Station Pizza\" on Meituan Takeaway with other pizza brands you've tried, and what makes \"Station Pizza\" stand out to you in terms of food quality?",
            "answer": " Yes, I've also ordered Big Pizza, but it's a bit expensive. One Big Pizza costs the same as two Station Pizzas, but the taste isn't much different."
        },
        {
            "question": "Can you elaborate on what specific aspects of the taste of Station Pizza and Big Pizza are similar, and are there any particular ingredients or cooking methods that you think contribute to this similarity, despite the price difference?",
            "answer": "Yes, they both have a very soft and fluffy texture, with lots of cheese. I think the amount of cheese and cooking time are very important as they greatly affect the taste."
        },
        {
            "question": "7. On a scale of 1-10, how would you rate the value for money of Station Pizza and Big Pizza, respectively, and can you explain your ratings?",
            "answer": "I rate Station Pizza 8 out of 10, and Big Pizza 7 out of 10, mainly due to the price difference."
        },
        {
            "question": "Can you compare the value for money of Station Pizza and Big Pizza to other food delivery services you've used in the past, and how do they rank in terms of value for money in your opinion?",
            "answer": " In terms of value for money, I think it's more cost-effective to dine at Big Pizza's restaurant with a group of people than at Station Pizza's restaurant."
        },
        {
            "question": "Can you elaborate on what specific factors contribute to your perception of value for money when dining at Big Pizza's restaurant versus Station Pizza's restaurant, and how do these factors influence your decision to choose one over the other?",
            "answer": "I think service, taste, and variety of pizzas all have an impact, and taste is also very important."
        },
        {
            "question": "Can you walk me through a specific instance where the taste of the pizza played a significant role in your decision to choose between Big Pizza's restaurant and Station Pizza's restaurant, and how did the taste experience influence your overall satisfaction with the meal?",
            "answer": "I can't quite remember anymore; it's been a long time since I've been to both of these restaurants at the same time."
        },
        {
            "question": "Here is the 11th question based on the interview outline and the user's previous question and answer:\n\n\"Can you describe a recent instance where you ordered pizza from Meituan Takeaway, and how did the taste experience influence your overall satisfaction with the meal? For example, was there a particular aspect of the pizza that you enjoyed or disliked, and did it affect your decision to reorder from the same restaurant or try a different one?\"",
            "answer": "The speed of making the pizza and the amount of cheese determine whether I will order again."
        },
        {
            "question": "\"Can you elaborate on what you mean by the 'amount of cheese' being a determining factor in your decision to reorder from the same restaurant? Is it a specific type of cheese, a certain quantity, or a particular way it's melted that makes a difference to you? For example, have you ever encountered a pizza with too little cheese, and if so, how did that affect your overall satisfaction with the meal?\"",
            "answer": "Yes, I have encountered that before. Some pizzas have very little cheese, and they taste dry. The quality of the cheese is also important; good cheese has a smoother texture."
        },
        {
            "question": "Can you describe a specific instance where the quality of the cheese affected your overall satisfaction with the meal, and how did that experience influence your decision to reorder from that restaurant or not?",
            "answer": "I can't remember either. Please move on to the next question."
        },
        {
            "question": "Here is the 14th question based on the interview outline and the user's previous question and answer:\n\n\"Moving on to the delivery experience, can you walk me through a recent instance where you had an issue with the delivery speed or service attitude of Meituan Takeaway couriers? For example, was the food delayed, or did the courier not follow your special instructions? How did you feel about the experience, and did it affect your overall satisfaction with Meituan Takeaway?\"",
            "answer": "Yes, the delivery person's attitude is very important. Some delivery persons have a bad temper. Normally, I don't mind delays, but if the delivery person has a bad temper, I can't tolerate it."
        },
        {
            "question": "\"That's really insightful. It sounds like the delivery person's attitude can greatly impact your experience with Meituan Takeaway. Can you tell me more about what specifically happens when you encounter a delivery person with a bad temper? For example, have they been rude to you, or have they not followed your instructions? How do you think Meituan Takeaway could improve their courier service to ensure that customers have a better experience?\"",
            "answer": "Yes, sometimes the delivery person can't find the address, or when we're in the park, they often have trouble finding us. Then they become impatient and angry, which makes for a poor experience."
        },
        {
            "question": "Here is the 16th question based on the interview outline and the user's previous question and answer:\n\n\"Can you walk me through a specific instance where the delivery person had trouble finding your address or location, and how they reacted to the situation? For example, what did they say or do that made you feel uncomfortable or frustrated? How did you handle the situation, and what do you think Meituan Takeaway could do to prevent such incidents from happening in the future?\"",
            "answer": "The delivery drivers are in a hurry to deliver the food. If they call twice and still can't reach you, they might say things like \"Can you hurry up?\" I think Meituan Waimai could use GPS to track both the user and the delivery driver, so the delivery driver knows the user's location, which would solve this problem."
        },
        {
            "question": "17th question: \"That's a great idea about using GPS to track both the user and the delivery driver. Can you elaborate on how you think this feature would improve the overall delivery experience for you? For example, would it reduce anxiety or frustration, or would it make you feel more in control of the delivery process? Additionally, are there any other benefits you think this feature would bring to the table?\"",
            "answer": "Having both the delivery driver and the user track each other can make us feel secure and in control at the same time."
        },
        {
            "question": "18th question: \"That's interesting that you mention feeling secure and in control with the GPS tracking feature. Can you think of any other scenarios or situations where having more visibility or transparency into the delivery process would be beneficial to you? For example, would you want to receive real-time updates on the status of your order, or would you want to be able to communicate directly with the delivery driver if there's an issue with the delivery?\"",
            "answer": "Yes, I hope the order status updates in real-time. Haha."
        },
        {
            "question": "19th question: \"That's a great point about wanting real-time order status updates. Can you walk me through a specific scenario where you felt frustrated or anxious about not knowing the status of your order, and how real-time updates would have made a difference in that situation?\"",
            "answer": "The real-time location update feature for delivery drivers on this mini program is already very good."
        },
        {
            "question": "\"You mentioned that the real-time location update feature for delivery drivers on the mini program is already very good. Can you elaborate on how this feature has positively impacted your overall experience with Meituan Takeaway, and are there any other aspects of the ordering process where you think real-time updates would be beneficial?\"",
            "answer": "Yes, the real-time location update feature affects many users, and precise positioning is even more important."
        }
    ],
}

访谈的报告模版

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
Meituan Takeaway User Experience Interview Result Report Framework
Interview Background: Through this interview , we hope to gain an in-depth understanding of users' experiences with Meituan Takeaway, so that we can better optimize products and services in the future, improving user satisfaction and experience.
Interview Objective: To better collect feedback on users' experiences with Meituan Takeaway, in order to optimize and improve the service in the future.
Results Abstract:
Summary of Key Findings:  xxxxxxxxxxxxx
Recommendations:  xxxxxxxxxxxxx
Next Steps:  xxxxxxxxxxxxx

Results Information:
1. Respondent Demographics
Age Distribution:
o20-30: 8 respondents、xx%
o30-40: 10 respondents、xx%
o40-50: 2 respondents、xx%
o50+ : xxx
Gender:
oMale: 16 respondents
oFemale: 4 respondents
Occupation:
oStudents: 6 respondents、xx%
oOffice workers: 10 respondents、xx%
oFreelancers: 4 respondents
Hobbies and Interests:
oTechnology: 12 respondents、xx%
oFood: 15 respondents、xx%
oFitness: 8 respondents、xx%
2. Usage Patterns
Frequency of Use:
oLess than 7 times per week: 1 respondents、xx%
o7-10 times per week: 12 respondents、xx%
o10-20 times per week: 6 respondents、xx%
oMore than 20 times+ per week: 2 respondents、xx%
Usage Scenarios:
oLunch: 15 respondents、xx%
oDinner: 18 respondents、xx%
oLate-night snacks: 10 respondents、xx%
oxx
Product Preferences:
oMeals: 18 respondents、xx%
oSnacks: 12 respondents、xx%
oBeverages: 8 respondents、xx%
3. Satisfaction Levels
Overall Satisfaction:
oAverage rating: 8/10
oRatings distributed 
Advantages:
oConvenience: 16 respondents、xx%、 xxxx
oVariety of food options: 14 respondents、xx%、 xxxx
oUser-friendly app interface: 12 respondents、xx%、 xxxx
Disadvantages:
oOccasional delays: 8 respondents 、xx%
oHigh delivery fees during peak hours: 10 respondents、xx%
4. Specific Experience Points
Order Process:
oSimple: 17 respondents、 xxxx
oIssues in customization: 3 respondents、 xxxx
Delivery Experience:
oSatisfied with speed: 15 respondents、xxxx 
oSatisfied with courier attitude: 16 respondents、 xxxx
oReported delays: 5 respondents、 xxxx
Food Quality:
oSatisfied: 17 respondents
oComplaints about temperature and freshness: 3 respondents、 xxxx
After-Sales Service:
oPositive feedback on refunds: 10 respondents、 xxxx
oSlow complaint resolutions: 6 respondents、 xxxx
App Experience:
oTechnical issues: 5 respondents、 xxxx
oOperational inconveniences: 4 respondents、 xxxx
5. User Habits and Behavior
Pricing Perception:
oAcceptable: 14 respondents
oExpensive : 6 respondents
Promotional Activities:
oInfluence on usage frequency: 12 respondents
6. Suggestions and Expectations
Areas for Improvement:
oLower delivery fees: xxxx
oFaster complaint resolution:xxx
oBetter order tracking:xxx
Desired Features:
oMore diverse food options: xx
oReal-time courier tracking: xx
oImproved loyalty rewards: x
7. Additional Feedback 
Other Comments:

Appendix
Interview Outline: List of all questions asked during the interviews.




Send the Result Report        Download Results Report


分析和生成

每个报告的模板分为题目，简介和段落详细信息，每个段落又分为几个要点，每个要点需要对所有访谈统计后的来，我们尝试了直接让LLM使用所有访谈结果，然后生成每个段落的统计要点，这是失败的，因为LLM不善于统计，尤其从如此众多的形式各异的回答中，按一定要求进行统计。

我们需要分步进行，一共分成4步。

1.构建表字段

根据报告模版，对每个段落的详细信息进行字段设计，让LLM生成构建的表格，例如下面，模版段落中需要对年龄分布进行统计，所以，我们需要收集每个访谈的年龄，所以设计的表的字段包括名称，描述，类型，默认值（有的人没有告诉年龄）

1
2
3
4
5
6
7
"part_template": """Age Distribution: eg: age distribution, maximum age, minimum age, etc.""",
"fields": [{
    "name": "Age",
    "description": "The interviewee's age, default -1",
    "type": "int",
    "default": -1,
}]

下面是对性别的示例，比年龄多了1个选项，性别必须是合理值，例如有的人回答的是中文的”男”，那么需要转换成”Male”

1
2
3
4
5
6
7
8
9
10
{
    "part_template": """Gender Distribution:""",
    "fields": [{
        "name": "Gender",
        "description": "Male or Female, default unknown",
        "type": "str",
        "default": "unknown",
        "options": ["Male","Female"]
    }]
},

完成报告中所需的所有字段，形成1个访谈表，每个用户都有1个相同的访谈表，包括上面的所有字段，例如性别，年龄，职业，爱好，外卖使用频率（按周统计），场景，食物偏好等等。

2.信息抽取

使用LLM和所有的访谈结果，根据设计好的要填充的表格，遍历每个访谈和每个表格的每个字段，逐个让LLM抽取出关键的字段内容，然后保存到每个用户的表中，这里我们收集了23份访谈结果，一个用户表有26个字段需要统计，那么至少需要调用23*26=598次LLM，耗费时间较长，可以考虑并发加速。PS：尝试过同时抽取多个字段，因为需要抽取后自动保存到表中，同时抽取效果不太好。

3.表内容整理

LLM抽取出来填入数据库表中的数据是比较混乱的，需要统一的整理后才能使用，例如抽取出下面的职业数据,我们可以看到用户的回答是多种多样的，相同的职业也可以回答出数种形式，那么我们就需要进行统一整理，保证相同的职业用相同的名称，这样才方便使用代码进行分析和统计。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
"name": "Occupation",
       "description": "eg: Student, Engineer, default unknown",
       "type": "str",
       "default": "unknown",
       "values": [
           "Data analyst",
           "数据分析师",
           "",
           "",
           "Programmer",
           "a programmer",
           "分析师",
           "程序员",
           "程序员",
           "a data analyst",
           "office worker",
           "professional service provider",
           "front-end development engineer",
           "female",
           "data analyst",
           "office worker",
           "front-end development engineer",
           "Office Worker",
           "软件开发工程师",
           "software development engineer",
           "分析师",
           "分析师",
           "analyst"
       ],

经过LLM整理后的数据如下，我们发现，基本已经分成了几类，而且语种也进行了统一。我们需要对所有字段都进行相同的处理。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
"clean_values": [
           "Data analyst",
           "Data analyst",
           "unknown",
           "unknown",
           "Programmer",
           "Programmer",
           "Data analyst",
           "Programmer",
           "Programmer",
           "Data analyst",
           "Office Worker",
           "Office Worker",
           "Front-end Development Engineer",
           "Office Worker",
           "Data analyst",
           "Office Worker",
           "Front-end Development Engineer",
           "Office Worker",
           "Software Development Engineer",
           "Software Development Engineer",
           "Data analyst",
           "Data analyst",
           "Data analyst"
       ]

4.报告生成

现在我们可以根据报告模版生成具体的报告了，这里我们更加细分成3个部分，首先根据模版的段落细节，使用LLM编写代码，调用所有用户的表，查询和统计，生成每个段落的答案，然后根据的出的段落答案按要求进行绘图，最后根据所有段落的答案，生成总结性摘要，完成整个报告模版。最后展示即可。

相关prompt

信息抽取prompt:

1
2
3
Read the interview records below, extract the required field content, and output it. If not found, output "XX field not mentioned in the interview records. 
                Required field is: {one_field}
                Interview records: {interview_text}

抽取后保存数据的prompt:

1
2
3
4
Here are the key information extracted from an interview record. Please write code to call a built-in function to save the key information. If there is no key information to output, there is no need to save it.
                key information field is: {one_field}
                example insert code: {example_insert_code}
                {api_tools_prompt}

数据整理的prompt,这里需要注意，我们要求LLM整理出新的数据输出和解释，因为你即使不要求LLM整理新数据的解释，它也会解释，而且经验会出现臆想，新数据变多，变少或者和原数据大变样。

1
2
3
4
5
The following is a data list. Please organize it, ensure the data types are correct, and unify the values as much as possible. For example, replace different words with the same meaning with a single term. Output the data list in its original format, maintaining the same number of entries. Do not reorder or Delete any element.If an element in the list severely deviates from the data description, change it to default value:'{info["default"]}'.
                            {data_define}
                            Please output in the following format:
                            New Data List: xxx
                            Explain: yyy

每个段落的细节答案的prompt

1
2
3
Complete the following statistical requirements based on the given data.
Statistical requirements: {part_template}
{data_define}

生成报告摘要的prompt:

1
2
3
We are currently conducting an interview abstract. Based on the following abstract references and the full text, generate a new abstract.
            abstract references: {topic_result["abstract"]}
            full text: {full_text}

UI展示

选择分析报告的主题

根据报告模版自动生成的数据表

根据访谈自动抽取的数据

数据清洗

数据分析

自动生成报告

结论

由于分步处理过程较多，而且LLM容易出现幻想，为了更精确的生成报告，所以需要对每个分布处理的过程进行监督，防止出现错误传导，假如抽取结果时，LLM就错了，那么后面的步骤也就没有意义了。

访谈结果报告生成
https://johnson7788.github.io/2024/07/16/%E8%AE%BF%E8%B0%88%E7%BB%93%E6%9E%9C%E6%8A%A5%E5%91%8A%E7%94%9F%E6%88%90/
作者
Johnson
发布于
2024年7月16日
许可协议