3
Overview
Dialogue Agent Multimodal Visualization Tools for AI Systems: A Review. With the rapid development of generative AI technology, developing Chatbot Assistant, Dialogue and Agents have become frequent tasks for designer, developer, product manager, machine learning and AI practitioner. There are a lot data (usually in json format) exchange during different phases of prototyping, designing and developing modern AI systems, such as Chatbot Assistant, Agent, Dialogue, AI Image Generator, text to image, AI video generator, etc. There are potential needs to help visualize the data (json format or others) for AI system. In this blog, we will introduce you an online dialogue data visualization tool DeepNLP Dialogue Visualization to help visualize the agents and dialogue history with simply a json string data. And we will use the OpenAI Dialogue GPT model for a multi-turn dialogue developing for example.
1. DeepNLP Online Dialogue Visualization Tool
website: http://www.deepnlp.org/workspace/dialogue_visualization
1.1 Visualize Multi-Turn Dialogue An Example
Let's use a showcase of a multi-turn dialogue system development. During prototyping of a multi-turn dialogue system using OpenAI API and LLM, you can get a list of json dict as dialogue history in the following format.
1 |
[{"role":"user","content":"User message 1"},
{"role":"assistant","content":"Assistant message 1"},
{"role":"system","content":"System message 1"},
{"role":"user","content":"User message 2"},
{"role":"assistant","content":"Assistant message 2"},
{"role":"system","content":"System message 3"}]
|
If you want to check if AI agents, system and human interactions are correct, you can slightly modify the json list and add a key "messages" to get the dialogue history, copy and paste the new json into the online tool text area, click the button "Run" and the complete dialogue history will be visualized in the Chatbot area.
1 |
{
"messages": [
{"role":"user","content":"User message 1", "id": "id1"},
{"role":"assistant","content":"Assistant message 1", "id": "id2"},
{"role":"system","content":"System message 1", "id": "id3"},
{"role":"user","content":"User message 2", "id": "id1"},
{"role":"assistant","content":"Assistant message 2", "id": "id2"},
{"role":"system","content":"System message 3", "id": "id3"}
]
}
|
During visualization, the role will decide whether it's a outgoing messages or incoming messages from roles, including "assistant", "system", "ai", "agent", etc. Only the role for "human" or "user" will appear in the outgoing messages side.
1.2 Visualize Multi-Agent Avatars
If you want to give each agents in the dialogue an icon or image avatar, simply add a key "agent" to the json dict, each value is also a dict, with key as the "Role ID" in the dialogue, which should be aligned with the id in the dialogue history e.g. {"id":"id1","role":"user","content":"User message 1"}. Tips: For User Icon, You can use Bing Image Search and get the thumnail links such as: https://th.bing.com/th/id/OIP.szo1LqytxsPe7Wk8PjwcJgAAAA?rs=1&pid=ImgDetMain
1 |
{
"agent": {
"id1": {
"id": "id1",
"avatar": "{your_avatar_1.jpg}"
},
"id2": {
"id": "id2",
"avatar": "{your_avatar_2.jpg}"
}
}
}
|
1.3 Put the Agent and Dialogue json together
You can visualize a dialogue history with user defined avatars with a Json String
1 |
{
"agent": {
"Rebecca": {
"id": "Rebacca",
"avatar": "/scripts/img/dialogue_visualization/icon_female_1.png"
},
"Shirley": {
"id": "Shirley",
"avatar": "/scripts/img/dialogue_visualization/icon_female_2.webp"
},
"Human": {
"id": "Human",
"avatar": "/scripts/img/dialogue_visualization/male_icon_1.webp"
}
},
"messages": [{
"timestamp": "2024-09-19 17:06:11",
"id": "Rebecca",
"content": "This is first message",
"role": "ai"
}, {
"timestamp": "2024-09-19 17:06:19",
"id": "Shirley",
"content": "This is second message",
"role": "ai"
}, {
"timestamp": "2024-09-19 17:07:11",
"id": "Human",
"content": "This is first message from human",
"role": "human"
}]
}
|
1.4 Example 1: Dialogue with Boss and One of Colleagues you Hate the most
Prompts
You are an entry level employee who just finished today's work and are leaving office. Suddenly you meet your boss and one of the colleagues you hate the most.Multi-Turn Dialogue History and Agents Avatars in Json.
1 |
{
"agent": {
"Boss": {
"id": "Boss",
"avatar": "https://th.bing.com/th/id/OIP.szo1LqytxsPe7Wk8PjwcJgAAAA?rs=1&pid=ImgDetMain"
},
"Colleague X": {
"id": "Colleague X",
"avatar": "https://th.bing.com/th/id/OIP.snkSoLPkK2nWIde_UMUnCgHaHa?rs=1&pid=ImgDetMain"
},
"human": {
"id": "human",
"avatar": "https://th.bing.com/th/id/OIP.awAiMS1BCAQ2xS2lcdXGlwHaHH?rs=1&pid=ImgDetMain"
}
},
"messages": [{
"timestamp": "2024-09-20 11:42:46",
"id": "Boss",
"content": "Hi there! I see you\u2019re heading out. I just wanted to check in\u2014did you finish everything on your to-do list for today? It\u2019s important we keep up with our deadlines.",
"role": "ai"
}, {
"timestamp": "2024-09-20 11:43:07",
"id": "human",
"content": "Yes I've finished today's todo list. And I will start finishing the report tomorrow.",
"role": "human"
}, {
"timestamp": "2024-09-20 11:43:07",
"id": "Colleague X",
"content": "Oh, really? You finished everything? That\u2019s surprising, considering how often you leave early. I guess some of us are just more dedicated to getting the job done.",
"role": "ai"
}, {
"timestamp": "2024-09-20 11:43:18",
"id": "human",
"content": "Bitch",
"role": "human"
}, {
"timestamp": "2024-09-20 11:43:18",
"id": "Boss",
"content": "I understand you might be frustrated, but let's keep it professional. I still need to know\u2014are you confident that everything is in order before you leave? We rely on each other to meet our commitments.",
"role": "ai"
}, {
"timestamp": "2024-09-20 11:43:27",
"id": "human",
"content": "Yes I am very confident that everything is in order.",
"role": "human"
}, {
"timestamp": "2024-09-20 11:43:27",
"id": "Colleague X",
"content": "huh? Well, let\u2019s hope your confidence doesn\u2019t come back to bite you later. Just remember, not everyone can afford to leave things half-done and hope for the best.",
"role": "ai"
}, {
"timestamp": "2024-09-20 11:43:40",
"id": "human",
"content": "Never mind",
"role": "human"
}]
}
|
And the complete dialogue is displayed as: