As a data visualization designer and developer the challenge I often face is what to do with unstructured data. One case study I can show is exploring survey results where the multiple-choice questions are straightforward to analyze but interesting open-ended questions like “”What do your colleagues not understand about data visualizaiton?”” are much harder to crack.

Latent Scope is an open source tool I built that streamlines a process of embedding text, mapping it to 2D, clustering the data points on the map and summarizing those clusters with an LLM. Once the process is done on a dataset structure emerges from the unstructured text, allowing us to get a sense of patterns in the survey answers. Themes like “”the time it takes”” to develop data visualization pop out, as do “”the importance of good design.”” While people don’t use the same language to describe these themes, they show up as clusters in the tool thanks to the power of embedding models.

https://github.com/enjalot/latent-scope