Nick Mullen | Digital Communications team blog: Recent Episodes
None
I recently spent an entertaining afternoon exploring new ways to visualise our web traffic data. If you’ve ever braved the wilds of Google Analytics, you’ll know it can be like trekking through a dense forest of numbers. It’s powerful, no doubt—but not easy on the eyes. Bar charts and pie graphs can only go so far, especially when trying to communicate your findings to someone who doesn’t speak fluent “data”.
Inspired by this excellent video from a data visualiser at Uber, I set out on a mission: build a 3D visualisation that brings web traffic to life over time.
Step 1 – Wrangling the data
I began by pulling a day’s worth of pageview data from our GA4 API. It seemed straightforward—until I saw just how much traffic even a slow day could produce.
Step 2 – Cleaning house
Next, I wrote a quick script to remove any page with fewer than three views. This helped slim the dataset down to something my laptop could handle without bursting into flames.
Step 3 – Colour coding
I then mapped pageviews to a greyscale—black for low traffic, white for high. Since our homepage receives significantly more traffic than any other page, I applied a logarithmic scale to avoid ending up with one blinding white dot surrounded by a field of nearly identical greys.
Step 4 – Into the 3rd dimension
Here’s where things got a bit tricky. I needed to convert URL paths into 3D coordinates. After trialling a few methods, I landed on using UMAP with 3D embedding. This grouped similar pages near each other, which turned out to be quite useful. I then scaled and centred the coordinates around the homepage—which, in hindsight, may not have been the wisest move.
Step 5 – Plotting with Plotly
Once I had my coordinates, it was time to plot. Plotly made it relatively straightforward to turn the processed data into an interactive 3D visualisation.
Step 6 – Adding the animation
Initially, I visualised just a single day. But things got far more interesting when I expanded the project to cover a full year—animating the data so that one second of animation represented one day of pageviews.
The Results: It worked (sort of)
The UMAP clustering grouped related pages together in a surprisingly elegant way. You could see the structure of the site—sections naturally forming their own little gravitational clusters.
One moment stood out in particular: the day the website went down. The animation didn’t just show a dip—it was a dramatic drop-off, instantly recognisable.
Was it perfect? Not quite. The scaling and centring introduced a sort of shaky-camera effect that made it a bit nauseating to watch all the way through. If I’d had more time, I would have worked on stabilising the animation for a smoother viewing experience.
Final thoughts
This was a proper nerdy side quest—but one that made me rethink how we present and explore web data. Turning flat analytics into a living, breathing visualisation isn’t just fun—it’s also a brilliant way to spot patterns and anomalies you’d otherwise miss.
Next up? Holograms.
The post Diving into web traffic: A 3D data visualisation experiment first appeared on Digital Communications team blog.
Web Developer Nick Mullen explains how he developed an AI-driven tool to support web content editors in Schools and professional services units (PSUs).
AI-powered tools are becoming increasingly prevalent in society, offering immense potential to enhance workflows and solve complex problems. Like many others, I am fascinated by the opportunities AI presents and have been exploring ways to leverage its capabilities for my organisation. In this article, I’ll share how I developed an AI-driven tool to support our content editors and improve their processes.
The challengeIn large organisations like ours, content editors play a vital role in creating and managing online content, often alongside their other responsibilities. This work is not easy. Content editing has become increasingly complex due to evolving legislation, changing guidelines, and the need for subject matter expertise. Content coordinators – colleagues in Schools and Units who edit webpages – are further challenged by their infrequent engagement with content editing tasks, limiting opportunities to practise and hone their skills.
To address these challenges, I set out to create a tool that would empower our content editors by providing AI-driven support for reviewing and improving text. My goal was not only to simplify their work but also to ensure that the University’s content adhered to established standards and best practices.
AI-powered content review toolI developed an AI-powered application that analyses and suggests improvements to text entered by users. The tool was trained on the University’s unique content requirements, including:
- specific rules and preferences governing grammar, tone and style
- best practices for writing accessible and engaging content tailored for online audiences
- ensuring that content is inclusive and adheres to accessibility requirements
In addition to formalised rules, I incorporated unwritten principles of content management, such as:
- maintaining a single source of truth to avoid redundancy
- promoting inclusive language across all communications
- encouraging content to be placed on the appropriate platform or page, such as linking to a person’s profile instead of duplicating personal details elsewhere
By combining these elements, I aimed to create a tool that not only reviews grammar but also enforces tone, accuracy, accessibility and organisational best practices.
How it worksThe tool was built using OpenAI’s API, which I chose for its versatility and familiarity with its functionality. Here’s a breakdown of the implementation process.
Data preparation:* I compiled a list of house style rules, creating multiple examples of correct and incorrect implementations for training purposes. * Accessibility and web-writing guidelines were also documented, along with unwritten rules like tone preferences and platform usage policies. * Together, these instructions formed a comprehensive framework for the AI model.
Layered AI processing:A layered approach was used to maximise accuracy and relevancy. Instead of relying on a single API request, I split the process into three sequential calls.
- Rewrite stage: the first API call rewrites the user’s input according to the University’s style, accessibility and content rules.
- Fact and tone check: the second call evaluates the rewritten text for factual accuracy and tone, ensuring consistency with University standards.
- Recommendation stage: the final call provides actionable suggestions without rewriting the text, for example, if the text describes an event, the tool recommends using the University’s event system.
Web integration:* The application is a lightweight web tool built with HTML, JavaScript and PHP. * Users interact with a simple web interface. Text entered into the tool is sent to a PHP function after a brief pause, triggering the sequential API calls. * Responses from the API are parsed and displayed to the user in an intuitive format, highlighting rewrites, corrections and recommendations.
Key technical considerations* Scalability: the modular nature of the PHP function and sequential API calls ensures that the tool can handle additional rules or features with minimal changes. * Custom training: while OpenAI’s API provides a general-purpose model, I customised its behaviour by embedding detailed instructions and examples relevant to our organisation. * Latency optimisation: the layered approach introduced additional API calls but ensured higher accuracy and adherence to standards. I optimised latency by batching some operations and caching frequently used rules. * User feedback loop: editors can provide feedback, helping refine its recommendations over time.
The above screenshot shows the application in action, applying our house style and correcting the capitalisation of “admissions Team”.
Impact and future plansThis AI-powered content review tool has the potential to significantly streamline the editing process for our team, reducing the cognitive load on Digital Communications content editors and ensuring consistent, high-quality content across our platforms. The application is still very much a working prototype.
Looking ahead, I plan to:
- improve the feedback given explaining what is wrong with the submitted text, highlighting the relevant rules and guidance
- incorporate machine learning to improve recommendations based on historical user feedback
- expand the tool’s scope to include additional features, such as suggesting SEO improvements or identifying redundant content across platforms
- explore deeper integrations with our CMS (Content Management System) for a seamless editing experience
This project has been a rewarding example of how AI can solve real-world problems while driving efficiency and quality in content management.
The post How I built a content review application first appeared on Digital Communications team blog.
Our team has recently been exploring the potential of AI in enhancing our users’ online experience. I have delved into the possibilities of using AI to develop customised web pages on demand. In this blog, I will cover the processes involved in building a proof of concept and the challenges faced in the implementation of such an application.
The idea behind this project was straightforward – to create a website capable of dynamically generating web pages based on the user’s input in the URL. For instance, if the user inputted /computing-research-subjects, the system would interpret the request as ‘construct a web page concerning the research subjects of the school of computing’. By leveraging our knowledge base, the AI would then construct a web page that precisely met the user’s request.
My approach consisted of the following five essential steps:
- Gathering data
- Assigning weight and labels
- Storing the data
- Customising and personalising
- Publishing and refining
Gathering dataCreating the knowledge base involved collecting a significant amount of content, which could have been sourced from various places, such as PDFs, databases, word documents, and websites. To simplify the process, I focused on content that was already published on our website. I ensured that the content was in its original form, without any formatting, styling, or images.
To accomplish this, I developed a basic web spider specifically designed to crawl our website. The spider targeted the main content of each page, extracting only the relevant text and disregarding other parts like headers, footers, and sidebars. The extracted text was then scrubbed of any embedded HTML and saved to a database.
Assigning weight and labelsDuring the cleanup process, I utilized the content headings and the first sentence within the text to generate metadata and assigned tags to the content to enhance its searchability.
Not all content carries the same importance. For instance, content that is linked from the homepage is generally more significant than content that is buried deep within the website. Thus, my idea was to incorporate web traffic statistics to determine the weighting of the content. This would involve assigning a higher weighting to pages that receive the most traffic, as opposed to pages with lower traffic. However, due to time constraints, I was unable to implement this weighting system and had to skip over it.
Storing the dataTo enable semantic search in my application, I decided to use vector embeddings instead of lexical search. This involved converting my text data into vector representations. Although I won’t dive into the technical details, there are many informative articles available online about embeddings and vectors. In simple terms, embeddings are a type of text encoding necessary for semantic search.
To perform the conversion, I utilized OpenAI’s embedding Application Programming Interface (API), however, there was a limitation on the amount of data I could send in a single request. Therefore, before calling the API, I needed to split my text into smaller chunks. To automate this process, I created a script that iterated through my database, dividing the text into manageable pieces and then converting each chunk into an embedding. These embeddings were then stored in a dedicated vector database.
It’s important to emphasize that, although my text resided in one database and the embeddings in another, these two sets of data were linked using a unique identifier. This linkage allowed me to maintain a connection between the embeddings and their corresponding text pieces.
Customising and personalisingThe process of customising and personalising content can vary for different individuals. For instance, a staff member seeking module information would require a different set of details than a student with the same purpose. By categorising both the content and the user, we can effectively deliver tailored information.
There are various approaches to categorising content, such as considering the user’s role, nationality, location (online or in-person), academic discipline, or even their residential hall at the University.
Additionally, personal preferences play a significant role. These preferences are specific to each individual user, taking into account their interests or dislikes, such as a strong inclination towards sports or a distaste for the colour blue. All of this information can be utilized to customise the presented content.
Regrettably, due to time constraints, I couldn’t delve further into this concept. However, I envision a potential process where the labelling conducted in step two can be combined with user information obtained from a login session or through cookies. This combination would then be used to exert influence on the content being delivered to the user.
Publishing and fine tuningThe final step involves integrating all the components. When a user submits a request, the system searches for relevant content and generates a web page to display to the user.
The complete workflow is as follows:
- The user submits a URL.
- The URL is converted into an embedding.
- The embedding is used to search the vector database.
- A subset of the text data (the search results) is sent to OpenAI, prompting the system to generate HTML content based on the provided subset.
- The generated HTML is displayed within a University-branded template.
During my experimentation with fine tuning in OpenAI, I aimed to train a model that can generate HTML content aligned with our University’s branding and content standards. I had limited success and ultimately utilized the basic ChatGPT 3.5 turbo model.
The navigation aspect worked exceptionally well. I instructed the AI to create navigation links that led to relevant pages based on the context of the current page, not to pages that already exist. As a result, when the user follows one of these links the entire process restarts with the new URL, effectively creating a dynamic website on the fly as the user navigates from page to page.
The drawbacksThere are a number of drawbacks to this approach:
- Slow user experience: building web pages on the fly requires API calls and database lookups that can be time consuming, resulting in a slow user experience.
- Batch processing limitations: the knowledge base is created using a batch process, meaning that any changes in content will only show up after the batch process has been run. This causes delays in updating the delivered content.
- Cost constraints: each request to OpenAI incurs a small cost, making running this process at scale significantly more expensive than a standard website.
- Quality of content: the quality of the generated content is dependent on the input data. Outdated or poor quality content can find its way into the knowledge base leading to the creation of subpar content. Utilizing AI to determine the quality of content before insertion could be beneficial.
- Lack of page layout customisation: efforts to customise page layouts based on content were unsuccessful. Every page ended up in the same standard template, removing the ability to adhere to design principles or format content for different delivery channels.
ConclusionDespite these challenges, I am satisfied with what has been built. There are still obstacles to overcome and improvements to be made, but I believe the foundational building blocks are in place for AI to revolutionise content delivery.
The post Using AI to dynamically generate web pages first appeared on Digital Communications team blog.
We have recently upgraded our search autocomplete functionality within the main site header. This update is designed to give users the best experience possible, reducing the amount of typing required and providing search results in fewer steps.
In addition, we have integrated voice-enabled search into the search field. By clicking on the microphone icon, users can now use their device’s microphone to search, without the need for typing. This is perfect for those who prefer to speak their queries to their device rather than type them out.
The search autocomplete is actually a combination of two searches. The first is a Funnelback search that generates suggested search terms based on the text that has been entered. If a user selects one of the suggested terms, a full search is triggered. The second is a search of our main collection, which is made up of most of our online content. If a user selects a result from this collection, they are taken directly to the web page, without the need for a search results page.
This is just the first in a series of updates to the search autocomplete. In future releases, we plan to expand the range of data that is searched and add additional functionalities.
The post Search autocomplete first appeared on Digital Communications team blog.
All of our academic Schools websites have a number of web pages that display people profile information. These web pages are automatically generated by combining data from multiple sources and applying a standard layout.
The following information is intended for staff members wishing to update their profile pages. The information on people pages is publicly accessible and has been optimised for searching. As such, any content published via people profile pages must be appropriate and relevant to the University.
It can take time for any change to cascade through our systems, please wait 24 hours before reporting an issue following an update.
Updating your informationIf you would like to update your information please consult the following guide so your request can be efficiently dealt with.
For changes to the following fields please contact HR directly by requesting a change to HR data to itservicedesk@st-andrews.ac.uk. Any change to this information will be subject to HR approval:
- title
- name
- job title.
The following data elements are taken from the Pure research portal. This information should be updated within Pure. For a full guide to updating your Pure information please consult the School webpage guide
- alternative name, known as
- research outputs
- biography
- research overview
- teaching activity
- profile keywords.
Keep all text formatting to a minimum and ensure it is not copied and pasted from MS Word. In that way the content will appear as intended.
Please ensure that all the information is correctly formatted with valid mark-up and adheres to our accessibility standards. If you are unsure of our accessibility standards, please contact itservicedesk@st-andrews.ac.uk
Some additional information is held within Terminal 4, this includes:
- rooms
- available times.
Please contact your schools computing office to update data within Terminal 4. Alternatively you can contact the Digital communications team by submitting a support call, requesting a change to availability by emailing itservicedesk@st-andrews.ac.uk.
Updating your profile imageYour profile images can be updated by going to Staff photos or using the link in your Pure profile. Please ensure that your profile image adheres to Digital Communications guidance.
Removing your profile pageIf you don’t wish to have your information displayed on the website please send a Remove my profile page request to itservicedesk@st-andrews.ac.uk.
Adding additional profilesAdditional profile pages can be requested for supplementary staff members such as visiting academics. If you would like to set up a profile page for a visiting academic please send request an additional profile page to itservicedesk@st-andrews.ac.uk.
The post Updating people pages first appeared on Digital Communications team blog.
Office 365 offers a number of different applications that can be used to save, share and organise your documents. Managing your files can get a bit confusing, after all, these applications are built on the same technology, have many of the same features and look very similar. In this article, I am going to look…
The post Teams, OneDrive SharePoint what is the difference? first appeared on Digital Communications team blog.
One question we are often asked is why a particular URL is not listed at the top of our search results. The University uses Funnelback to provide our website search functions. The search works by scanning our documents and giving each document a score (ranking). The score reflects the overall quality of the content, the…
The post Funnelback search rankings first appeared on Digital Communications team blog.
One of the biggest challenges we face is to ensure that our websites continue to work on all manner of devices. To meet this challenge, many web developers have adopted a mobile first design process. The core principle of mobile first is that the design process should begin by first designing for the smallest screen…
The post Offline first first appeared on Digital Communications team blog.
In this article, I will discuss the process of adding cards, labels, custom types and lists to Trello using php and the Trello API. The team have recently completed a requirements gathering exercise, capturing stakeholder needs and wants for our careers redevelopment project. We interviewed over 50 people and collected over 150 user stories. Each…
The post Adding cards with the Trello API first appeared on Digital Communications team blog.
We have recently revamped the University’s module catalogue to take advantage of a number of Funnelback features, delivering a more sophisticated search algorithm and returning higher quality results. In this blog post, I am going to discuss the steps taken to set up the key search features used in the module catalogue. This article is…
The post Searching the module catalogue with Funnelback first appeared on Digital Communications team blog.
It’s no secret that I love a good set of requirements. Working from a solid set of requirements makes my life so much easier. They give clarity over what is needed, formally setting the scope of the work, and provide a focal point for conversation. I am always wary of projects with ambiguous open-ended requirements.…
The post What makes for a good set of requirements? first appeared on Digital Communications team blog.
Setting up a project correctly is an important but often overlooked phase of any project. Successful intranet-redesign projects start before sketching, feature selection, or coding. In light of an upcoming internal content project, the team have been conducting some research into successful intranets. In doing so, I found an interesting article about ‘User-Centered Intranet Redesigns‘.…
The post How to set up an intranet project in 11 steps first appeared on Digital Communications team blog.
Back in November 2019, I attended the Squiz Summit in Edinburgh. The summit is an annual event that brings together Squiz staff with their customers, to share in their experiences of using Squiz products. For the past year, the University has been using Squiz’s Funnelback search functionality to power our main site search. The event…
The post Squiz Summit 2019 first appeared on Digital Communications team blog.
The digital communications team have reviewed the University’s academic Schools website traffic to determine what the most popular parts of the websites are. Using data captured by Google Analytics, we reviewed one years worth of data from the following School websites: English Modern Languages Mathematics and Statistics Chemistry Art History. We aggregated all the visited web…
The post Top areas of school websites first appeared on Digital Communications team blog.
The following article is intended to provide a technical overview for an administrator of T4 with access to their community extranet site. One of the current team objectives is to amalgamate a number of research related blogs into one centralised WordPress blog. The University’s research pages within the main website (published by T4) are to…
The post Displaying WordPress content within TerminalFour first appeared on Digital Communications team blog.
Hi, my name is Nick. On 15 January, I became the latest member to join the digital communications team. As a web developer, it will be my job to support the delivery of the University’s web projects and to assist in the maintenance of the University’s websites. I am joining the team from Fife Council,…
The post New team member first appeared on Digital Communications team blog.