Light Blue Touchpaper: Recent Episodes
None
On 2 October, TU Delft are starting a new online three course series on cybersecurity economics. I am co-teaching this course with Michel van Eeten (TU Delft), Daniel Woods (University of Edinburgh), Simon Parkin (TU Delft), Rolf van Wegberg (TU Delft), Tyler Moore (Tulsa Uni) and Rainer Böhme (Innsbruck Uni). The course also features content from Ross Anderson (University of Cambridge), recorded before his passing. Ross was passionate about teaching, and was deeply involved in the design of this MOOC.
The first course on Foundation and Measurement provides you with foundational micro-economic concepts to explain security behavior of various actors involved securing the organization – internally, like IT and business units, and externally, like suppliers, customers and regulators. Next, it equips you with a causal framework to understand how to measure the effectiveness of security controls, as well as what measurements are currently available.
The second course on Users and Attackers presents a wealth of insights on the individuals involved in security: from user behavior to the strategies of attackers. Contrary to popular opinion, users are not the weakest link. If you want to know why do users not follow company security policies, you need to look at the costs imposed on them. On the side of the attackers, there are also clear incentives at work. The course covers the latest insights on attacker behavior.
The third course on Solutions covers answers to overcome the incentive misalignment and information problems at the level of organizations and at the level of markets. Starting with the standard framework of risk management, the course unpacks how to identify solutions in risk mitigation and risk transfer and where risk acceptance might be more rational. Finally, we need to address market failures, since they end up undermining the security of firms and society at large.
TL;DRTwo invitations to Cambridge (UK):
2025-03-25: the Rossfest Symposium, in honour of Ross Anderson (1956-2024)
https://www.cl.cam.ac.uk/events/rossfest/
2025-03-26 and 27: the 29th Security Protocols Workshop
https://www.cl.cam.ac.uk/events/spw/2025/
Start writing, and sign up here for updates on either or both:
https://forms.gle/Em9Hy43aRqrdGmd17
Rossfest Symposium25 March 2025The Rossfest Symposium and its posthumous Festschrift is a celebration and remembrance of our friend and colleague Ross Anderson, who passed away suddenly on 28 March 2024, aged 67.
Ross Anderson FRS FRSE FREng was Professor of Security Engineering at the University of Cambridge and lately also at the University of Edinburgh. He was a world-leading figure in security. He had a gift for pulling together the relevant key people and opening up a new subfield of security research by convening a workshop on the topic that would then go on to become an established series, from Fast Software Encryption to Information Hiding, Scrambling for Safety, Workshop on Economics and Information Security, Security and Human Behavior and so forth. He co-authored around 300 papers. His encyclopedic Security Engineering textbook (well over 1000 pages) is dense with both war stories and references to research papers. An inspiring and encouraging supervisor, Ross graduated around thirty PhD students. And as a contagiously enthusiastic public speaker he inspired thousands of researchers around the world.
The Rossfest Symposium is an opportunity for all of us who were touched by Ross to get together and celebrate his legacy.
The Festschrift volumeScientific papersWe solicit scientific contributions to a posthumous Festschrift volume, in the form of short, punchy papers on any security-related topic. These submissions will undergo a lightweight review process by a Program Committee composed of former PhD students of Ross:
- Joseph Bonneau, New York University
- Richard Clayton, University of Cambridge
- Markus Kuhn, University of Cambridge
- Tyler Moore, University of Tulsa
- Ilia Shumailov, Google DeepMind
- Frank Stajano, University of Cambridge
Accepted papers will be published in the Festschrift book and presented at the event. For a subset of the accepted papers, the authors will be invited to submit an expanded version to a special issue of the Journal of Cybersecurity honouring Ross’s scholarly contributions and legacy.
Submissions are limited to five pages in LNCS format (we did say short and punchy!) and will get an equally short presentation slot at the Rossfest. Let’s keep it snappy, as Ross himself would have liked. Five pages excluding bibliography and any appendices, that is, and maximum eight pages total.
Topic-wise, anything related to security, taking the word in its broadest sense, is fair game, from cryptography and systems to economics, psychology, policy and much more, spanning the wide spectrum of fields that Ross himself explored over the course of his career. But make it a scientific contribution rather than just an opinion piece.
Authors will grant us a licence to publish and distribute their articles in the Festschrift but will retain copyright and will be able to put their articles on their web pages or resubmit them wherever else they like. We won’t ask for article charges for publishing in the Festschrift. Bound copies of the Festschrift volume will be available to purchase at cost during the Rossfest Symposium, or later through print-on-demand. A DRM-free PDF will be available online at no charge.
Informal memoriesWe also solicit informal “cherished memories” contributions along the lines of those collected by Ahn Vu at anderson.love. These too will be collected in the volume and a selection of them will be presented orally at the event.
The Rossfest SymposiumThe Rossfest Symposium will last the whole day and will take place at the Computer Laboratory (a.k.a. the Department of Computer Science and Technology of the University of Cambridge), where Ross taught, researched and originally obtained his own PhD. Street address: 15 JJ Thomson Avenue, Cambridge CB3 0FD, UK.
Attendance at the Rossfest Symposium is free and not conditional on the submission of a contribution, but registration will be required for us to manage numbers and catering.
In the evening there shall also be a formal celebration banquet at Trinity College. To attend, please purchase a ticket. Registration and payment links shall appear on this page in due course. Street address: Trinity Street, Cambridge CB2 1TQ, UK.
We have timed the Rossfest to be adjacent in time and space to the Security Protocols Workshop, an event that Ross regularly attended. The SPW will take place in Trinity College Cambridge on 26 and 27 March 2025. This will allow you to attend both events with a single trip to Cambridge. Note that attendance at SPW requires presenting a position paper: unlike the Rossfest, at SPW all attendees must also speak.
Accommodation in CambridgeThe chosen dates are out of term, meaning you might be able to book a room in one of the 31 colleges through www.universityrooms.com. Otherwise, consider www.airbnb.com, www.booking.com, www.expedia.com or your favourite online booking aggregator.
Sign upTo receive notifications (e.g. “the registration and payment links are now up”), sign up on this Google form. Self-service unsubscribe at any time.
Dates25 November 2024: Deadline for submission of Festschrift articles
23 December 2024: Invitations to authors to present orally
13 January 2025: Early bird (discounted) registration deadline for banquet
10 February 2025: Final registration deadline for banquet and symposium
25 March 2025: Rossfest Symposium (and optional banquet)
26-27 March 2025: Security Protocols Workshop (unrelated but possibly of interest)
Security Protocols Workshop26 and 27 March 2025The Twenty-ninth International Workshop on Security Protocols will take place from Wednesday 26 March to Thursday 27 March 2025 in Cambridge, United Kingdom. It will be dedicated to the memory of Ross Anderson and preceded by the Rossfest Symposium, which will take place on Tuesday 25 March 2025, also in Cambridge, UK. Come to both!
As in previous years, attendance at the International Workshop on Security Protocols is by invitation only. (How do I get invited? Submit a position paper.)
ThemeThe theme of the 2025 workshop is: “Controversial Security – In honour of Ross Anderson”. In other words, “any security topic that Ross Anderson might have wanted to debate with you”, which leaves you with plenty of leeway.
This is a workshop for discussion of novel ideas, rather than a conference for finished work. We seek papers that are likely to stimulate an interesting discussion. New authors are encouraged to browse through past volumes of post-proceedings (search for Security Protocols Workshop in the Springer LNCS series) to get a flavour for the variety and diversity of topics that have been accepted in past years, as well as the lively discussion that has accompanied them.
DetailsThe long-running Security Protocols Workshop has hosted lively debates with many security luminaries (the late Robert Morris, chief scientist at the NSA and well known for his pioneering work on Unix passwords, used to be a regular) and continues to provide a formative event for young researchers. The post-proceedings, published in LNCS, contain not only the refereed papers but the curated transcripts of the ensuing discussions (see the website for pointers to past volumes).
Attendance is by invitation only. To be considered for invitation you must submit a position paper: it will not be possible to come along as just a member of the audience. Start writing now! “Writing the paper is how you develop the idea in the first place”, in the wise words of Simon Peyton-Jones.
The Security Protocols Workshop is, and has always been, highly interactive. We actively encourage participants to interrupt and challenge the speaker. The presented position papers will be revised and enhanced before publication as a consequence of such debates. We believe the interactive debates during the presentations, and the spontaneous technical discussions during breaks, meals and the formal dinner, are part of the DNA of our workshop. We encourage you to present stimulating and disruptive ideas that are still at an initial stage, rather than “done and dusted” completed papers of the kind that a top-tier conference would expect. We are interested in eliciting interesting discussion rather than collecting archival material.
SubmissionsShort indicative submissions are preferred. You will have the opportunity to extend and revise your paper both before the pre-proceedings are issued, and again after the workshop. At the workshop, you will be expected to spend a few minutes introducing the idea of your paper, in a way that facilitates a longer more general discussion. Pre-proceedings will be provided at the workshop. See the Submission page for more details.
Committee• Fabio Massacci (Program Chair), University of Trento / Vrije Universiteit Amsterdam
• Frank Stajano (General Chair), University of Cambridge
• Vashek (Vaclav) Matyas, Masaryk University
• Jonathan Anderson, Memorial University
• Mark Lomas, Capgemini
Accommodation in CambridgeThe chosen dates are out of term, meaning you might be able to book a room in one of the 31 colleges through www.universityrooms.com. Otherwise, consider www.airbnb.com, www.booking.com, www.expedia.com or your favourite online booking aggregator.
Dates25 November 2024: Submission of position papers
23 December 2024: Invitations to authors
13 January 2025: Early bird (discounted) registration deadline
3 February 2025: Revised papers due
10 February 2025: Final registration deadline
25 March 2025: Rossfest Symposium (unrelated but possibly of interest)
26-27 March 2025: Security Protocols Workshop
For further details visit the web page at the top of this message. To be notified when the registration and paper submission pages open, , sign up on this Google form. Self-service unsubscribe at any time.
The seventeenth Security and Human Behavior workshop was hosted by Bruce Schneier at Harvard University in Cambridge, Massachusetts on the 4th and 5th of June 2024 (Schneier blog).
This playlist contains audio recordings of most of the presentations, curated with timestamps to the start of each presentation. Click the descriptions to see them.
On the lunch of the first day, several attendees remembered the recently departed Ross Anderson, who co-founded this workshop with Bruce Schneier and Alessandro Acquisti in 2008. That recording is in the playlist too.
Kami Vaniea kept up Ross’s tradition by liveblogging most of the event.
I’ll be hosting next year’s SHB at the University of Cambridge.
The Cambridge Cybercrime Centre‘s seventh one day conference on cybercrime was held on Monday, 10th June 2024.
Similar to previous “liveblog” coverage of conferences and workshops on Light Blue Touchpaper, here is a “liveblog”-style overview of the talks at this year’s conference.
L. Jean Camp – Global Cyber Resilience Using a Public Health Model of eCrime (Keynote)
Who gets phished? This still hasn’t changed much in 20 years. We still don’t know how people are targeted, or even if they are targeted. People need to identify security indicators, domain names, etc., and this is hard. Current practice with warnings does not provide what people need. While people can learn how to use bad interfaces, we can’t expect people to pay attention all the time and without interruption. Expertise alone is not adequate: LastPass devs were phished. She looked at phishing factors, and asked how good each population was at identifying phishing and legitimate websites, finding familiarity and gender did not have a significant difference for phishing websites, but found familiarity was important for identifying legitimate websites. Later, they asked participants about security expertise. We tend to write warnings for ourselves (security experts), rather than for end users. They also compared risk perception across populations. Overall, they found computer expertise (positive) and age (negative) were the primary factors in identifying phishing pages. How can we learn from public health to provide more effective warnings which work for the wider general population?
Gabriella Williams – Beyond Borders: Exploring Security, Privacy, Cultural, and Legal Challenges in Metaverse Sexual Harassment
PhD researcher in digital identity and age assurance methods to mitigate against virtual harms. The virtual reality environment (metaverse) has new risks and harms, by creating a new environment with anonymity where people can be whoever they want to be. Gabriella asks if sexual harassment is a crime in the metaverse? There is no legal framework currently, and there are varying jurisdictions online. Metaverse has cultural issues, with standing close to someone, making unwanted contact, and inappropriate jokes. How can this be moderated? Lots of issues with collecting metadata on social interactions, biometric data, and security issues with over reliance on automation and threats to authentication and integrity. Their current research is looking at challenges around implementing age assurance, and how identities can be authenticated.
Bomin Keum – The Incel Paradox: Does Collective Self-Loathing Facilitate Radicalisation or Belonging?
What don’t we know and why don’t we know it? We have a hard time agreeing on what radicalisation is, but this is a process rather than instances of extremist violence. Online radicalisation is facilitated through anonymity, perceived strength in numbers, and too much information spread and absorbed quickly. Bomin considers the use of the Us vs Them framework: collectively constructed perception differentiating the in-group from the out-group. Incel communities show negativity within the group as well as out, which is different to other communities. The Us vs Them framework has “us” as self-directed victimhood with men deprived of their “right to sex” whereas the “them” refers to a perception of society giving “too much freedom to women”. What are the self and other narrative framings, and which topics are associated with self vs other narrative frames? Bomin compares 2019 and 2020 datasets around the start of the pandemic. Internal group themes have helplessness and victimisation, whereas outside has unfair advantages and shameful other. Collectively, there are narratives of community, violence, and vision. They note you can’t take discussions at face value, as the language used can be quite extreme and text-level analysis may not reflect intent. Also, there is some shifting from blame to mockery of others. Not all radical actors commit violence but can inform facilitators behind intensification. Applying theories to these communities can be questionable, due to the unique aspects of the communities, and needs further data-driven research to improve on theory.
Jason Nurse – Ransomware Harms and the Victim Experience
Supply chain issue with St. Thomas’ Hospital last week, where a supplier of hospitals was hit by ransomware, and a critical incident was declared in London. Focus in the media on the financial impact, but what are the other harms of this, on both individuals and society? Jason carried out a literature review, and ran workshops and interviews alongside harm modelling to explore effects. What do we know already from the literature, and what can we learn from individuals? Interviews were focused on people who were subject to a ransomware attack or had professional experience of supporting organisations affected by ransomware. This includes cyber insurance organisations, which are now a big player. Gathering qualitative data from interviews, and using thematic analysis. Findings show this is a serious risk for all organisations, including small businesses: “everything you relied on yesterday doesn’t work today”. Can also create reputational harm for organisations. Applying the idea of orders of harm: first-order are harms directly to the person or org, second-order are downstream orgs and individuals, and third-order are the economy and society. Implications include a loss of trust in law enforcement, reduced faith in public services, and normalisation of cybercrime. Other impacts include harms to staff: staff members having to deal with the situation, including overworking to resolve issues. Highlights potential correlations between burnout and cybersecurity issues. Next, Jason looks at how to model harms. They gather data on well publicised events and to establish relationships between harms. This finds many downstream harms: we can more deeply explore harms arising throughout society rather than just “the data was encrypted”.
Ethan Thomas – Operation Brombenzyl and Operation Cronos
DDoS for hire continues to be a threat, enabling easy attacks against infrastructure, and these are targeted by site take downs and arrests. Finding a new way to provide a longer lasting impact, disrupting the marketplace. Using splash pages to deter users, and also creating law enforcement-run DDoS for hire websites. Some of the disguised sites were “seized”, others were “outed” as NCA controlled, and some are still running. Second operation is Cronos, again using deception but applied to ransomware attacks. Finding broad deterrence messaging doesn’t always work well, now there is focus on showing victims cases where cybercriminals did not uphold their promises.
Luis Adan Saavedra del Toro – Sideloading of Modded Apps: User Choice, Security and Piracy
What are modded apps, and why do users use them? Android users have the capability of installing any app they download from the internet, outside of the Google Play Store. Third-party stores have ads and user review features. Modded apps have unlocked pro features, such as a modded Spotify app to bypass ads and other paid features. Modded gaming apps have free in-app purchases. Luis found over 400 modded Android app markets, and crawled the 13 most popular, creating the ModZoo dataset. Most of these modded apps are games, and lots of duplicates across markets. None of the markets had any payment infrastructure. They discovered apps with changed code had added additional permissions and advertising libraries. Some apps with Ad IDs had been changed. 9% of those with modded code were malicious. iOS has misconceptions around jailbreaking. iOSModZoo has ~30k apps. iOSZoo is a dataset of ~55k free App Store apps. Most iOS modded apps are pirated copies of paid apps.
Felipe Moreno-Vera and Daniel S. Menasché – Beneath the Cream: Unveiling Relevant Information Points from CrimeBB with Its Ground Truth Labels
Looking at exploits which are shared on underground forums. The team used three types of labels: post-type, intent, and crime-type, which they used to complement their approach to tracking keywords, their usage, and different vulnerability levels discussed. They create a classifier for threats, so they can identify what is being discussed. They use regex to identify CVEs, and a function to identify language. They note the labels used were only available for one site, and later use ChatGPT to create more labels for posts. They find ChatGPT improves on existing labels.
Jeremy D. Seideman, Shoufu Luo, and Sven Dietrich – The Guy In The Chair: Examining How Cybercriminals Use External Resources to Supplement Underground Forum Conversations
“Guy in the chair” is the support network that “connects the dots”. They looked at underground forum conversations to identify what this support network is. Do people post URLs, do they advertise things, do they talk about other communications? What is the wider context? Past literature shows that forums work best as a social network, forming communities. Their project examines the use of offensive AI usage, presenting their data pipeline, which they use to clean data prior to using topic transfer models. Following this, they identified buckets of URLs. The majority of known links were other forums, code sharing, image hosting, and file sharing. Lots of the links had link rot. Future work will further explore the application of analysis methods used with archaeological count data to their dataset.
Anh V. Vu – Yet Another Diminishing Spark: Low-level Cyberattacks in the Israel-Gaza Conflict
Anh notes differing perspectives of cyberwar in the world media, with a strong focus on high-profile cyber attacks. However, what is happening with low-level cybercrime actors and the services supporting these attacks? They are using data from website defacement attacks and UDP amplification DDoS attacks, alongside collections of volunteer hacking discussions. They contrast the conflicts of Russia vs Ukraine and Israel vs. Gaza. Anh finds interest in low-level DDoS and defacement attacks dropped off quickly, although notes that these findings should not be confounded with state-sponsored cyber attacks.
Dalyapraz Manatova – Relationships Matter: Reconstructing the Organisational Structure of a Ransomware Group
Dalyapraz has been studying dynamics of cybercrime networks, thinking about these as a socio-technical complex system, with technical, economical, and social factors. Existing literature shows that eCrime has “communities”, with admins and moderators. When these communities are disrupted, they often move to other places. Participants often have different pseudonyms for who they are communicating with, e.g. as an administrator or to trade. However, these communities are more like organisations, with roles, tasks, scale, scope. Follows a similar structure to aaS services.
Marilyne Ordekian – Investigating Wrench Attacks: Physical Attacks Targeting Cryptocurrency Users
Wrench attacks have been around since the start of Bitcoin, yet have received little academic attention. Marilyne gathered data on wrench attacks through Bitcoin Talk discussions and interviews. Incidents were reported across different areas, from 2011 to 2021. There were peaks of incidents, which coincided with bitcoin reaching an all-time high. Why? Potential reasons include financial gain, theft is easier than hacking, and no account transfer limits. They found that 25% of these incidents occurred during in-person meet ups. Are wrench attacks reported? No, they are underreported. They propose safety mechanisms for individuals, including not bragging, diversifying of funds, and digital safety practices. Also, they suggest existing regulations could be strengthened, such as improved KYC verification to consider the risk of wrench attacks. System design changes could include redesigning apps to hide balance amounts.
Mariella Mischinger – Investigating and Comparing Discussion Topics in Multilingual Underground Forums
Mariella finds prior literature on forums is often missing understanding of the content and does not find niche topics. Also, there is a lack of research into multilingual underground forums, and a lack of data on invite-only forums. Datasets contain lots of noisy, informal language. They found sentence embeddings were useful to cluster the content into topics, as this included the context and intention in sentences. They extracted topics from the clusters of sentences using LDA, with cosine similarity then finding similar topics across languages. Mariella then finds this method can be used to find pockets of knowledge: topics only discussed in one language. Further work identified dark keywords, combining neologisms with groups of keywords.
Someone else will undoubtedly say it much better than I will here but one of us has to break the very sad news: Ross Anderson died yesterday.
His enthusiasm, his wide-spectrum intellectual curiosity and his engaging prose were unmatched. He stood up vigorously for the causes he believed in. He formed communities around the new topics he engaged with, from information hiding to fast software encryption, security economics, security and human behaviour and more. He served as an inspiring mentor for generations of graduate students at Cambridge—I know first hand, as I was fortunate enough to be admitted as his PhD student when he was still a freshly minted lecturer and had not graduated any students yet. I learnt my trade as a Cambridge Professor from him and will be forever grateful, as will dozens of my “academic brothers” who were also supervised by him, several of whom post regularly on this blog.
Ross, thank you so much for your lively, insightful and stimulating contributions to every subfield of security. You leave a big void that no one will be able to fill. I will miss you.
RIP
In 2008, I wrote a blog to introduce J-PAKE, a password-authenticated key exchange (PAKE) protocol (joint work with Peter Ryan). The goal of that blog was to invite public scrutiny of J-PAKE. Sixteen years later, I am pleased to say that no attacks on J-PAKE have been found and that the protocol has been used in many real-world applications, e.g., Google Nest, ARM Mbed, Amazon Fire stick, Palemoon sync and Thread products.
J-PAKE is a balanced PAKE, meaning that both sides must hold the same secret for mutual authentication. In the example of the J-PAKE-based IoT commissioning process (part of the Thread standard), a random password is generated to authenticate the key exchange process and is discarded afterwards. However, in some cases, it is desirable to store the password. For example, in a client-server application, the user knows a password, while the server stores a one-way transformation of the password.
PAKE protocols designed for the above client-server setting are called augmented (as opposed to the balanced in the peer-to-peer setting). So far the only augmented PAKE protocol that has enjoyed wide use is SRP-6a, e.g., used in Apple’s iCloud, 1Password and Proton mail. SRP-6a Is the latest version of the 1998 SRP-3 scheme due to Wu after several revisions to address attacks. Limitations of SRP-6a are well known, including heuristic security, a lack of efficiency (due to the mandated use of a safe prime as the modulus) and no support for elliptic curve implementation.
In 2018, an augmented PAKE scheme called OPAQUE was proposed by Jarecki, Krawczyk and Xu. In 2020, IETF selected OPAQUE as a candidate for standardization. A theoretical advantage promoted in favour of OPAQUE is the so-called pre-computation security. When the server is compromised, an offline dictionary attack to uncover the plaintext password is possible for both OPAQUE and SRP-6a. For OPAQUE, its pre-computation security means that the attacker can’t use a pre-computed table, whilst for SRP-6a, the attacker may use a pre-computed table, but it must be a unique table for each user, which requires a large amount of computation and storage. Therefore, the practical advantage provided by pre-computation security is limited.
Apart from pre-computation security, OPAQUE has a few open issues which leave it unclear whether it will replace SRP-6a in practice. First, the original OPAQUE protocol defined in the 2018 paper leaks password update information to passive attackers, whilst SRP-6a doesn’t have this leakage. Furthermore, OPAQUE relies on a constant-time hash-to-curve function available for all elliptic curves, but details about the instantiation of this function remain to be established. Finally, the 2018 paper didn’t give a full specification of OPAQUE. In 2020, when OPAQUE was selected by IETF, its specification remained incomplete. The task of completing the spec was left as a post-selection exercise; today, it is still not finished.
Motivated by the recognised limitations of SRP-6a and OPAQUE, we propose a new augmented PAKE scheme called Owl (joint work with Samiran Bag, Liqun Chen and Paul van Oorschot). Owl is obtained by efficiently adapting J-PAKE to an augmented setting, providing the augmented security against server compromise with yet lower computation than J-PAKE. To the best of our knowledge, Owl is the first augmented PAKE solution that provides systematic advantages over SRP-6a in terms of security, computation, round efficiency, message sizes, and cryptographic agility.
On 5 March 2024, I gave a presentation on Owl at Financial Cryptography and Data Security 2024 (FC’24) in Curacao. The purpose of this blog is to invite public scrunity of Owl. See the Owl paper and the FC slides for further details. An open-source Java program that shows how Owl works in an elliptic curve setting is freely available. We hope security researchers and developers will find Owl useful, especially in password-based client-server settings where a PKI is unavailable (hence TLS doesn’t apply). Same as J-PAKE, Owl is not patented and is free to use.
Britain’s National Crime Agency has spent the last five years trying to undermine encryption, saying it might stop them arresting hundreds of men every month for downloading indecent images of children. Now they complain that most of the men they do prosecute escape jail. Eight in ten men convicted of image offences escaped an immediate prison sentence, and the NCA’s Director General Graeme Biggar describes this as “striking”.
I agree, although the conclusions I draw are rather different. In Chatcontrol or Child Protection? I explained how the NCA and GCHQ divert police resources from tackling serious contact offences, such as child rape and child murder, to much less serious secondary offences around images of historical abuse and even synthetic images. The structural reasons are simple enough: they favour centralised policing over local efforts, and electronic surveillance over community work.
One winner is the NCA, which apparently now has 200 staff tracing people associated with alarms raised automatically by Big Tech’s content surveillance, while the losers include Britain’s 43 local police forces. If 80% of the people arrested as a result of Mr Biggar’s activities don’t even merit any jail time, then my conclusion is that the Treasury should cut his headcount by at least 160, and give each Chief Constable an extra 3-4 officers instead. Frontline cops agree that too much effort goes into image offences and not enough into the more serious contact crimes.
Mr Biggar argues that Facebook is wicked for turning on end-to-end encryption in Facebook Messenger, as won’t be able to catch as many bad men in future. But if encryption stops him wasting police time, well done Zuck! Mr Biggar also wants Parliament to increase the penalties. But even though Onan was struck dead by God for spilling his seed upon the ground, I hope we can have more rational priorities for criminal law enforcement in the 21st century.
On Tuesday we had a seminar from Liz Fong-Jones entitled “Reverse engineering hate” about how she, and a dozen colleagues, have been working to take down a hate speech forum called Kiwi Farms. We already published a measurement study of their campaign, which forced the site offline repeatedly in 2022. As a result of that paper, Liz contacted us and this week she told us the inside story.
The forum in question specialises in personal attacks, and many of their targets are transgender. Their tactics include doxxing their victims, trawling their online presence for material that is incriminating or can be misrepresented as such, putting doctored photos online, and making malicious complaints to victims’ employers and landlords. They describe this as “milking people for laughs”. After a transgender activist in Canada was swatted, about a dozen volunteers got together to try to take the site down. They did this by complaining to the site’s service providers and by civil litigation.
This case study is perhaps useful for the UK, where the recent Online Safety Bill empowers Ofcom to do just this – to use injunctions in the civil courts to take down unpleasant websites.
The Kiwi Farms operator has for many months resisted the activists by buying the services required to keep his website up, including his data centre floor space, his transit, his AS, his DNS service and his DDoS protection, through a multitude of changing shell companies. The current takedown mechanisms require a complainant to first contact the site operator; he publishes complaints, so his followers can heap abuse on them. The takedown crew then has to work up a chain of suppliers. Their processes are usually designed to stall complainants, so that getting through to a Tier 1 and getting them to block a link takes weeks rather than days. And this assumes that the takedown crew includes experienced sysadmins who can talk the language of the service providers, to whose technical people they often have direct access; without that, it would take months rather than weeks. The net effect is that it took a dozen volunteers thousands of hours over six months from October 22 to April 23 to get all the Tier 1s to drop KF, and over $100,000 in legal costs. If the bureaucrats at Ofcom are going to do this work for a living, without the skills and access of Liz and her team, it could be harder work than they think.
Liz’s seminar slides are here.
People who write about cyber-conflict often talk of hacktivists and other civilian volunteers who contribute in various ways to a cause. Might the tools and techniques of cybercrime enable its practitioners to be effective auxiliaries in a real conflict? Might they fall foul of the laws of war, and become unlawful combatants?
We have now measured hacktivism in two wars – in Ukraine and Gaza – and found that its effects appear to be minor and transient in both cases.
In the case of Ukraine, hackers supporting Ukraine attacked Russian websites after the invasion, followed by Russian hackers returning the compliment. The tools they use, such as web defacement and DDoS, can be measured reasonably well using resources we have developed at the Cambridge Cybercrime Centre. The effects were largely trivial, expressing solidarity and sympathy rather than making any persistent contribution to the conflict. Their interest in the conflict dropped off rapidly.
In Gaza, we see the same pattern. After Hamas attacked Israel and Israel declared war, there was a surge of attacks that peaked after a few days, with most targets being strategically unimportant. In both cases, discussion on underground cybercrime forums tailed off after a week. The main difference is that the hacktivism against Israel is one-sided; supporters of Palestine have attacked Israeli websites, but the number of attacks on Palestinian websites has been trivial.
I was delighted by two essays by Anton Howes on The Replication Crisis in History Open History. We computerists have long had an open culture: we make our publications open, as well as sharing the software we write and the data we analyse. My work on security economics and security psychology has taught me that this culture is not yet as well-developed in the social sciences. Yet we do what we can. Although we can’t have official conference proceedings for the Workshop on the Economics of Information Security – as then the economists would not be able to publish their papers in journals afterwards – we found a workable compromise by linking preprints from the website and from a liveblog. Economists and psychologists with whom we work have found their citation counts and h-indices boosted by our publicity mechanisms; they have incentives to learn.
A second benefit of transparency is reproducibility, the focus of Anton’s essay. Scholars are exposed to many temptations, which vary by subject matter, but are more tempting when it’s hard for others to check your work. Mathematical proofs should be clear and elegant but are all too often opaque or misleading; software should be open-sourced for others to play with; and we do what we can to share the data we collect for research on cybercrime and abuse.
Anton describes how more and more history books are found to have weak foundations, where historians quote things out of context, ignore contrary evidence, and elaborate myths and false facts into misleading stories that persist for decades. How can history correct itself more quickly? The answer, he argues, is Open History: making as many sources publicly available as possible, just like we computerists do.
As it happens, I scanned a number of old music manuscripts years ago to help other traditional music enthusiasts, but how can this be done at scale? One way forward comes from my college’s Archives Centre, which holds the personal papers of Sir Winston Churchill as well as other politicians and a number of eminent scientists. There the algorithm is that when someone requests a document, it’s also scanned and put online; so anything Alice looked at, Bob can look at too. This has raised some interesting technical problems around indexing and long-term archiving which I believe we have under control now, and I’m pleased to say that the Archives Centre is now celebrating its 50th anniversary.
It would also be helpful if old history books were as available online as they are in our library. Given that the purpose of copyright law is to maximise the amount of material that’s eventually available to all, I believe we should change the law to make continued copyright conditional on open access after an initial commercial period. Otherwise our historians’ output vanishes from the time that their books come off sale, to the time copyright expires maybe a century later.
My own Security Engineering book may show the way. With both the first edition in 2001 and the second edition in 2008, I put six chapters online for free at once, then released the others four years after publication. For the third edition, I negotiated an agreement with the publishers to put the chapters online for review as I wrote them. So the book came out by instalments, like Dickens’ novels, from April 2019 to September 2020. On the first of November 2020, all except seven sample chapters disappeared from this page for a period of 42 months; I’m afraid Wiley insisted on that. But after that, the whole book will be free online forever.
This also makes commercial sense. For both the 2001 and 2008 editions, paid-for sales of paper copies increased significantly after the whole book went online. People found my book online, liked what they saw, and then bought a paper copy rather than just downloading it all and printing out a thousand-odd pages. Open access after an exclusive period works for authors, for publishers and for history. It should be the norm.
There are many different ways to represent the same text in Unicode. We’ve previously exploited this encoding-visualization gap to craft imperceptible adversarial examples against text-based machine learning systems and invisible vulnerabilities in source code. In our latest paper, we demonstrate another attack that exploits the same technique to target Google Search, Bing’s GPT-4-powered chatbot, and … Continue reading How to Spread Disinformation with Unicode →
I’m at the 22nd Workshop on the Economics of Information Security in Geneva, and will be liveblogging the sessions in the followups to this post. Links to previous editions of WEIS, along with liveblogs and recordings, may be found here.
Recently I was contacted by a Falklands veteran who was a victim of what appears to have been a classic pre-play attack; his story is told here. Almost ten years ago, after we wrote a paper on the pre-play attack, we were contacted by a Scottish sailor who’d bought a drink in a bar in … Continue reading The Pre-play Attack in Real Life →
The Workshop on Security and Human Behaviour is happening right now in Carnegie-Mellon University and I’ll be liveblogging it in followups to this post. The participants’ papers are here, while the liveblogs and papers from previous workshops are here.
Until about now, most of the text online was written by humans. But this text has been used to train GPT3(.5) and GPT4, and these have popped up as writing assistants in our editing tools. So more and more of the text will be written by large language models (LLMs). Where does it all lead? What will happen to GPT-{n} once LLMs contribute most of the language found online?
And it’s not just text. If you train a music model on Mozart, you can expect output that’s a bit like Mozart but without the sparkle – let’s call it ‘Salieri’. And if Salieri now trains the next generation, and so on, what will the fifth or sixth generation sound like?
In our latest paper, we show that using model-generated content in training causes irreversible defects. The tails of the original content distribution disappear. Within a few generations, text becomes garbage, as Gaussian distributions converge and may even become delta functions. We call this effect model collapse.
Just as we’ve strewn the oceans with plastic trash and filled the atmosphere with carbon dioxide, so we’re about to fill the Internet with blah. This will make it harder to train newer models by scraping the web, giving an advantage to firms which already did that, or which control access to human interfaces at scale. Indeed, we already see AI startups hammering the Internet Archive for training data.
After we published this paper, we noticed that Ted Chiang had already commented on the effect in February, noting that ChatGPT is like a blurry jpeg of all the text on the Internet, and that copies of copies get worse. In our paper we work through the math, explain the effect in detail, and show that it is universal.
This does not mean that LLMs have no uses. As one example, we originally called the effect model dementia, but decided to rename it after objections from a colleague whose father had suffered dementia. We couldn’t think of a replacement until we asked Bard, which suggested five titles, of which we went for The Curse of Recursion.
So there we have it. LLMs are like fire – a useful tool, but one that pollutes the environment. How will we cope with it?
WEIS 2023, the 22nd Workshop on the Economics of Information Security, will be held in Geneva from July 5-7, with a theme of Digital Sovereignty. We now have a list of sixteen accepted papers; there will also be three invited speakers, ten posters, and ten challenges for a Digital Sovereignty Hack on July 7-8.
The deadline for early registration is June 10th, and we have discount hotel bookings reserved until then. As Geneva gets busy in summer, we suggest you reserve your room now!
Everyone’s worried that the UK Online Safety Bill and the EU Child Sex Abuse Regulation will put an end to end-to-end encryption. But might a law already passed by the EU have the same effect?
The Digital Markets Act ruled that users on different platforms should be able to exchange messages with each other. This opens up a real Pandora’s box. How will the networks manage keys, authenticate users, and moderate content? How much metadata will have to be shared, and how?
In our latest paper, One Protocol to Rule Them All? On Securing Interoperable Messaging, we explore the security tensions, the conflicts of interest, the usability traps, and the likely consequences for individual and institutional behaviour.
Interoperability will vastly increase the attack surface at every level in the stack – from the cryptography up through usability to commercial incentives and the opportunities for government interference.
Twenty-five years ago, we warned that key escrow mechanisms would endanger cryptography by increasing complexity, even if the escrow keys themselves can be kept perfectly secure. Interoperability is complexity on steroids.
A number of governments are trying to mandate surveillance software in devices that support end-to-end encrypted chat; the EU’s CSA Regulation and the UK’s Online Safety bill being two prominent current examples. Colleagues and I wrote Bugs in Our Pockets in 2021 to point out what was likely to go wrong; GCHQ responded with arguments about child protection, which I countered in my paper Chat Control or Child Protection.
As lawmakers continue to discuss the policy, the latest round in the technical argument comes from the Rephrain project, which was tasked with evaluating five prototypes built with money from GCHQ and the Home Office. Their report may be worth a read.
One contender looks for known-bad photos and videos with software on both client and server, and is the only team with access to CSAM for training or testing (it has the IWF as a partner). However it has inadequate controls both against scope creep, and against false positives and malicious accusations.
Another is an E2EE communications tool with added profanity filter and image scanning, linked to age verification, with no safeguards except human moderation at the reporting server.
The other three contenders are nudity detectors with various combinations of age verification or detection, and of reporting to parents or service providers.
None of these prototypes comes close to meeting reasonable requirements for efficacy and privacy. So the project can be seen as empirical support for the argument we made in “Bugs”, namely that doing surveillance while respecting privacy is really hard.
Back in 2015 I helped record a course in security economics in a project driven by colleagues from Delft. This was launched as an EDX MOOC as well as becoming part of the Delft syllabus, and it has been used in many other courses worldwide. In Brussels, in December, a Ukrainian officer told me they use it in their cyber defence boot camp.
There’s been a lot of progress in security economics over the past seven years; see for example the liveblogs of the workshop on the economics of information security here. So it’s time to update the course, and we’ll be working on that between now and May.
If there are any topics you think we should cover, or any bugs you’d like to report, please get in touch!
“Booters” (they usually call themselves “stressers” in a vain attempt to appear legitimate) are denial-of-service-for-hire websites where anyone can purchase small scale attacks that will take down a home Internet connection, a High School (perhaps there’s an upcoming maths test?) or a poorly defended business website. Prices vary but for around $20.00 you can purchase as many 10 minute attacks as you wish to send for the next month! In pretty much every jurisdiction, booters are illegal to run and illegal to use, and there have been a series of Law Enforcement take-downs over the years, notably in the US, UK, Israel and the Netherlands.
On Wednesday December 14th, in by far the biggest operation to date, the FBI announced the arrest of six booter operators and the seizure of 49 (misreported as 48) booter domain names. Visiting those domains will now display a “WEBSITE SEIZED” splash page.
The seizures were “evidence based” in that the FBI specifically targeted the most active booters by taking advantage of one of the datasets collected by the Cambridge Cybercrime Centre, which uses self-reported data from booters.
The majority of booters (c 70%) report the number of users they have and the number of attacks performed — this is part of their marketing… if others are using the booter at scale, then this will help convince a new visitor that the booter is not a scam and so they will purchase.
On the Monday before the FBI seizure there were 108 operational booters — of various sizes. The “top 20” were the only booters reporting more than 1000 attacks per day (on average over the previous week). On the Wednesday, seventeen of these were shut down.
booter boots/day 1 ( 1) stresser.app 23166 2 ( 7) blackstresser.net 10809 3 ( 3) brrsecurity.org 6672 4 ( 8) zerostresser.com 5641 5 ( 6) nightmarestresser.com 5003 6 ( 5) dragonstresser.com 4919 7 ( 9) sunstresser.com 3422 8 (19) defconpro.net 3118 9 (12) xxxxxxxxxxxx.xxx 288610 (10) stresser.top 268011 (14) yyyyyyyyyyyy.yyy 261612 (11) stresser.gg 245513 (15) kraysec.com 223814 (20) quantum-stresser.net 220715 (17) mcstorm.io 184316 (13) zdstresser.net 178917 (16) bootyou.net 173418 (22) dreams-stresser.io 165119 (18) zzzzzzzzzzzz.zzz 163820 (21) api-sky.xyz 1446
I am not naming #9 … but it wasn’t seized because it was a scam (several testing sessions failed to deliver any denial-of-service traffic at all). #11 is outside the FBI’s jurisdiction but local law enforcement is expected to act in the New Year, and #19 was not operational for several weeks and so it was never tested. The US judiciary would only hand down court orders for websites that had been determined to be working booters — taking money under false pretences is a matter for the Federal Trade Commission not the FBI. In fact #9 was far from alone in being tested and not working … people inclined to purchase booter services might reflect on the fact that unseized domains are where all the scams are to be found!
About half the booters websites have decided that is a Good Idea to resurrect themselves with new domain names. They are perhaps under the impression that it will be another four years before the FBI repeats a takedown (the last big action was in December 2018), but this does seem an unwise assumption to me. However, there is early evidence that publicity around the FBI’s action (assisted by advertising campaigns run by the British and Dutch police) has suppressed supply as well as demand.
The “top 10” chart for the booters reporting 1000+ attacks/day on Monday 26th (12 days on from the FBI action) looks like this … I have given the exact Dec 12th figures except when this was unrepresentative of recent levels of activity. As can be seen almost all of the booters are doing far less business than before — an overall reduction of about 50%. The full list runs to 75 booters (down from 108 2 weeks ago), but as I indicated above, perhaps half of these don’t actually work in practice.
booter boots/day Dec 12th 1 NEW name for stresser.app 12949 23166 2 NEW name for stresser.best 9066 usually 15000+ 3 NEW name for cyberstress.us 7659 usually 20000+ 4 NEW name for quantum-stresser.net 4470 2207 5 NEW name for zerostresser.com 3927 5641 6 zzzzzzzzzzzz.zzz 2814 1638 7 xxxxxxxxxxxx.xxx 1850 2886 8 NEW name for nightmarestresser.com 1766 5003 9 NEW name for dreams-stresser.io 1694 165110 vvvvvvvvvvvv.vvv 1578 usually 120011 wwwwwwwwwwww.www 1329 178912 NEW name for mcstorm.io 1074 184313 NEW name for stresser.gg 1056 245514 NEW name for redstresser.cc 1049 usually 1000also yyyyyyyyyyyy.yyy was running at around 5K attacks per day, but haderased some logs and reset the counters so I don't have an exact number.
When we studied the impact of the 2018 initiative in a 2019 IMC paper (Booting the Booters), we found that that it took around six weeks for activity to return to previous levels. This time around Law Enforcement is being provided with extremely timely evidence of the impact of what they are doing. Since they are taking action based on evidence, I am reasonably confident that the booter marketplace is going to disrupted for rather more than six weeks this time.
I’m hiring a Research Assistant/Associate to work on the EPSRC-funded Adaptive PETs to Protect & emPower People during Life Transitions (AP4L) project. The project is being undertaken with the Universities of Surrey, Queen Mary, Strathclyde, Edge Hill, and Edinburgh.
AP4L is a program of interdisciplinary research, centring on the online privacy & vulnerability challenges that people face when going through major life transitions. The four transitions we are considering in the scope of this project are relationship breakdowns; LBGT+ transitions or transitioning gender; entering/ leaving employment in the Armed Forces; and developing a serious illness or becoming terminally ill. Our central goal is to develop privacy-by-design technologies to protect & empower people during these transitions.
We are looking for a researcher with experience in quantitative data analysis, threat assessment, data science, machine learning and/or natural language processing, as well as excellent programming and technical writing skills. Expertise in cybercrime or privacy enhancing technologies (PETs) research is desirable, but not essential. Successful applicants will review the relevant literature, design research projects, develop tools, collect and analyse data, and write research outputs.
The role will analyse life transitions from the attacker’s perspective, such as how and where they gather information about their victims. This will require the analysis of cybercrime forums and similar data at scale. Furthermore, the tools we develop are designed for an adversarial context. Adversaries include those known to individuals, such as interfamilial abuse, as well as targeted and indiscriminate attacks. The researcher will also undertake a rigorous threat analysis for each of the tools developed within the overall project.
The full details are available here.
Today I publish a detailed rebuttal to the argument from the intelligence community that we need to break end-to-end encryption in order to protect children. This has led in the UK to the Online Safety Bill and in the EU to the proposed Child Sex Abuse Regulation, which has become known in Brussels as “chatcontrol”. … Continue reading Chatcontrol or Child Protection? →
Our latest paper demonstrates how a Trojan or backdoor can be inserted into a machine-learning model by the compiler. In his Turing Award lecture, Ken Thompson explained how this could be done to an operating system, and in previous work we’d shown you you can subvert a model by manipulating the order in which training … Continue reading ML models must also think about trusting trust →
The Department of Computer Science and Technology is hiring six new faculty members, including an Assistant or Associate Professor in the area of Privacy and/or Security. The Department is one of the world leaders in computer security, with outstanding historic contributions (such as the Needham-Schroeder protocol and the economics of computer security), as well as … Continue reading Assistant/Associate Professor in Security and Privacy →
Yesterday I took part in a panel discussion organised by the Adam Smith Institute on the Online Safety Bill. This sprawling legislative monster has outlasted not just six Secretaries of State for Culture, Media and Sport, but two Prime Ministers. It’s due to slither back to Parliament in November, so we wrote a Policy Brief … Continue reading The Online Safety Bill: Reboot it, or Shoot it? →
Talking Trojan: Analyzing an Industry-Wide Disclosure tells the story of what happened after we discovered the Trojan Source vulnerability, which broke almost all computer languages, and the Bad Characters vulnerability, which broke almost all large NLP tools. This provided a unique opportunity to measure software maintenance in action. Who patched quickly, reluctantly, or not at … Continue reading Talking Trojan: Analyzing an Industry-Wide Disclosure →
Online anonymous platforms such as forums enable freedom of speech, but also facilitate misogyny, extremism, and political polarisation. We have collected tens of millions of postings to such forums and created a new tool for social scientists to study how these phenomena are linked. Far-right extremism has been associated with a growing number of mass … Continue reading ExtremeBB: Supporting Large-Scale Research into Misogyny and Online Extremism →
Last year, we disclosed two related vulnerabilities that broke a wide range of systems. In our Bad Characters paper, we showed how to use Unicode tricks – such as homoglyphs and bidi characters – to mislead NLP systems. Our Trojan Source paper showed how similar tricks could be used to make source code look one … Continue reading The Dynamics of Industry-wide Disclosure →
In this blog post, we describe how we used rigorous engineering methods to provide high assurance of key security properties of CHERI architectures, with machine-checked mathematical proof, as well as to complement and support traditional design and development workflows, e.g. by automatically generating test suites. This shows that, by judicious use of rigorous semantics at design time, we can do much better than test-and-debug development.
Following last year’s row about Apple’s proposal to scan all the photos on your iPhone camera roll, EU Commissioner Johansson proposed a child sex abuse regulation that would compel providers of end-to-end encrypted messaging services to scan all messages in the client, and not just for historical abuse images but for new abuse images and … Continue reading Text mining is harder than you think →
I’m at the twenty-first Workshop on the Economics of Information Security, hosted by the University of Tulsa. I’ll be liveblogging the sessions in followups to this post. There’s a live stream for which you can register here.
Yesterday I was asked for advice on anonymously reporting a new crypto scam that a potential victim had spotted before they lost money (hint: to a first approximation all cryptocurrencies and cryptoassets are a scam). In the end they got fed up with the difficulty of finding someone they could tell and gave up. However, … Continue reading Reporting cybercrime is hard: NCA link to Action Fraud broken for 3 years →
The Workshop on Security and Human Behaviour is now an in-person event once more, after two years online. It’s happening in Cambridge and I’ll be liveblogging it in followups to this post.
Today, May 11, EU Commissioner Ylva Johannson announced a new law to combat online child sex abuse. This has an overt purpose, and a covert purpose. The overt purpose is to pressure tech companies to take down illegal material, and material that might possibly be illegal, more quickly. A new agency is to be set … Continue reading European Commission prefers breaking privacy to protecting kids →
A Research Assistant/Associate position is available at the Department of Computer Science and Technology to work on the ERC-funded Interdisciplinary Cybercrime Project (iCrime). We are looking to appoint a computer scientist to join an interdisciplinary team reporting to Dr Alice Hutchings. iCrime incorporates expertise from criminology and computer science to research cybercrime offenders, their crime … Continue reading Hiring for iCrime →
Gus Simmons is one of the pioneers of cryptography and computer security. His contributions to public-key cryptography, unconditional authentication, covert channels and information hiding earned him an honorary degree, fellowship of the IACR, and election to the Rothschild chair of mathematics when he visited us in Cambridge in 1996. And this was his hobby; his … Continue reading A striking memoir by Gus Simmons →
I have taken over the second-year Security course at Cambridge, which is traditionally taught in Easter term. From the end of April onwards I will be teaching three lectures per week. Taking advantage of the fact that Cambridge academics own the copyright and performance rights on their lectures, I am making all my undergraduate lectures … Continue reading Security course at Cambridge →
Whistleblowing is dangerous business. Whistleblowers face grave consequences if they’re caught and, to make matters worse, the anonymity set – the set of potential whistleblowers for a given story – is often quite small. Mass surveillance regimes around the world don’t help matters either. Yet whistleblowing has been crucial in exposing corruption, rape and other … Continue reading CoverDrop: Securing Initial Contact for Whistleblowers →
Professor Robert N. M. Watson (Cambridge), Professor Simon W. Moore (Cambridge), Professor Peter Sewell (Cambridge), Dr Jonathan Woodruff (Cambridge), Brooks Davis (SRI), and Dr Peter G. Neumann (SRI) After over a decade of research creating the CHERI protection model, hardware, software, and formal models and proofs, developed over three DARPA research programmes, we are at … Continue reading Arm releases experimental CHERI-enabled Morello board as part of £187M UKRI Digital Security by Design programme →
This week sees the start of a course on security engineering that Sam Ainsworth and I are teaching. It’s based on the third edition of my Security Engineering book, and is a first cut at a ‘film of the book’. Each week we will put two lectures online, and here are the first two. Lecture … Continue reading Security engineering course →
Electhical is an industry forum whose focus is achieving a low total footprint for electronics. It is being held on Friday December 10th at Churchill College, Cambridge. The speakers are from government, industry and academia; they include executives and experts on technology policy, consumer electronics, manufacturing, security and privacy. It’s sponsored by ARM, IEEE, IEEE … Continue reading Electhical 2021 →
The 2022 Workshop on the Economics of Information Security will be held at Tulsa, Oklahoma, on 21-22 June 2022. Paper submissions are due by 28 February 2022. After two virtual events we’re eager to get back to meeting in person if we possibly can. The program chairs for 2022 are Sadia Afroz and Laura Brandimarte, … Continue reading WEIS 2022 call for papers →
CHERI (Capability Hardware Enhanced RISC Instructions) is an architectural extension to processor Instruction-Set Architectures (ISAs) adding efficient support for fine-grained C/C++-language memory protection as well as scalable software compartmentalisation. Developed over the last 11 years at SRI International and the University of Cambridge, CHERI is now the subject of a £187M UK Industrial Strategy Challenge … Continue reading Report: Assessing the Viability of an Open-Source CHERI Desktop Software Ecosystem →
End-to-end (E2E) encryption is now widely deployed in messaging apps such as WhatsApp and Signal and billions of people around the world have the contents of their message protected against strong adversaries. However, while the message contents are encrypted, their metadata still leaks sensitive information. For example, it is easy for an infrastructure provider to tell which customers are communicating, with whom and when.
Anonymous communication hides this metadata. This is crucial for the protection of individuals such as whistleblowers who expose criminal wrongdoing, activists organising a protest, or embassies coordinating a response to a diplomatic incident. All these face powerful adversaries for whom the communication metadata alone (without knowing the specific message text) can result in harm for the individuals concerned.
Tor is a popular tool that achieves anonymous communication by forwarding messages through multiple intermediate nodes or relays. At each relay the outermost layer of the message is decrypted and the inner message is forwarded to the next relay. An adversary who wants to figure out where A’s messages are finally delivered can attempt to follow a message as it passes through each relay. Alternatively, an adversary might confirm a suspicion that user A talks to user B by observing traffic patterns at A’s and B’s access points to the network instead. If indeed A and B are talking to each other, there will be a correlation between their traffic patterns. For instance, if an adversary observes that A sends three messages and three messages arrive at B shortly afterwards, this provides some evidence that A talks to B. The adversary can increase their certainty by collecting traffic over a longer period of time.
Mix networks such as Loopix use a different design, which defends against such traffic analysis attacks by using (i) traffic shaping and (ii) more intermediate nodes, so called mix nodes. In a simple mix network, each client only sends packets of a fixed length and at predefined intervals (e.g. 1 KiB every 5 seconds). When there is no payload to send, a cover packet is crafted that is indistinguishable to the adversary from a payload packet. If there is more than one payload packet to be sent, packets are queued and sent one by one on the predefined schedule. This traffic shaping ensures that an observer cannot gain any information from observing outgoing network packets. Moreover, mix nodes typically delay each incoming message by a random amount of time before forwarding it (with the delay chosen independently for each message), making it harder for an adversary to correlate a mix node’s incoming and outgoing messages, since they are likely to be reordered. In contrast, Tor relays forward messages as soon as possible in order to minimise latency.
Mix Networks work well for pairwise communication, but we found that group communication creates a unique challenge. Such group communication encompasses both traditional chat groups (e.g. WhatsApp groups or IRC) and collaborative editing (e.g. Google Docs, calendar sync, todo lists) where updates need to be disseminated to all other participants who are viewing or editing the content. There are many scenarios where anonymity requirements meet group communication, such as coordination between activists, diplomatic correspondence between embassies, and organisation of political campaigns.
The traffic shaping of mix networks makes efficient group communication difficult. The limited rate of outgoing messages means that sequentially sending a message to each group member can take a long time. For instance, assuming that the outgoing rate is 1 message every 5 seconds, it will take more than 8 minutes to send the message to all members in a group of size 100. During this process the sender’s output queue is blocked and they cannot send any other messages.
In our paper we propose a scheme named Rollercoaster that greatly improves the latency for group communication in mix networks. The basic idea is that group members who have already received a message can help distribute it to other members of the group. Like a chain reaction, the distribution of the message gains momentum as the number of recipients grows. In an ideal execution of this scheme, the number of users who have received a message doubles with every round, leading to substantially more efficient message delivery across the group.
Rollercoaster works well because there is typically plenty of spare capacity in the network. At any given time most clients will not be actively communicating and they are therefore mostly sending cover traffic. As a result, Rollercoaster actually improves the efficiency of the network and reduces the rate of cover traffic, which in turn reduces the overall required network bandwidth. At the same time, Rollercoaster does not require any changes to the existing Mix network protocol and can benefit from the existing user base and anonymity set.
The basic idea requires more careful consideration in a realistic environment where clients are offline or do not behave faithfully. A fault-tolerant version of our Rollercoaster scheme addresses these concerns by waiting for acknowledgement messages from recipients. If those acknowledgement messages are not received by the sender in a fixed period of time, forwarding roles are reassigned and another delivery attempt is made via a new route. We also show how a single number can seed the generation of a deterministic forwarding schedule. This allows efficient communication of different forwarding schedules and balances individual workloads within the group.
We presented our paper at USENIX Security ‘21 (paper, slides, and recording). It contains more extensions and optimisations than we can summarise here. There is also an extended version available as a tech report with more detailed security arguments in the appendices. The paper reference is:
Daniel Hugenroth, Martin Kleppmann, and Alastair R. Beresford. Rollercoaster: An Efficient Group-Multicast Scheme for Mix Networks. Proceedings of the 30th USENIX Security Symposium (USENIX Security), 2021.
Today we are releasing Trojan Source: Invisible Vulnerabilities, a paper describing cool new tricks for crafting targeted vulnerabilities that are invisible to human code reviewers.
Until now, an adversary wanting to smuggle a vulnerability into software could try inserting an unobtrusive bug in an obscure piece of code. Critical open-source projects such as operating systems depend on human review of all new code to detect malicious contributions by volunteers. So how might wicked code evade human eyes?
We have discovered ways of manipulating the encoding of source code files so that human viewers and compilers see different logic. One particularly pernicious method uses Unicode directionality override characters to display code as an anagram of its true logic. We’ve verified that this attack works against C, C++, C#, JavaScript, Java, Rust, Go, and Python, and suspect that it will work against most other modern languages.
This potentially devastating attack is tracked as CVE-2021-42574, while a related attack that uses homoglyphs – visually similar characters – is tracked as CVE-2021-42694. This work has been under embargo for a 99-day period, giving time for a major coordinated disclosure effort in which many compilers, interpreters, code editors, and repositories have implemented defenses.
This attack was inspired by our recent work on Imperceptible Perturbations, where we use directionality overrides, homoglyphs, and other Unicode features to break the text-based machine learning systems used for toxic content filtering, machine translation, and many other NLP tasks.
More information about the Trojan Source attack can be found at trojansource.codes, and proofs of concept can also be found on GitHub. The full paper can be found here.
In August, Apple announced a system to check all our iPhones for illegal images, then delayed its launch after widespread pushback. Yet some governments continue to press for just such a surveillance system, and the EU is due to announce a new child protection law at the start of December.
Now, in Bugs in our Pockets: The Risks of Client-Side Scanning, colleagues and I take a long hard look at the options for mass surveillance via software embedded in people’s devices, as opposed to the current practice of monitoring our communications. Client-side scanning, as the agencies’ new wet dream is called, has a range of possible missions. While Apple and the FBI talked about finding still images of sex abuse, the EU was talking last year about videos and text too, and of targeting terrorism once the argument had been won on child protection. It can also use a number of possible technologies; in addition to the perceptual hash functions in the Apple proposal, there’s talk of machine-learning models. And, as a leaked EU internal report made clear, the preferred outcome for governments may be a mix of client-side and server-side scanning.
In our report, we provide a detailed analysis of scanning capabilities at both the client and the server, the trade-offs between false positives and false negatives, and the side effects – such as the ways in which adding scanning systems to citizens’ devices will open them up to new types of attack.
We did not set out to praise Apple’s proposal, but we ended up concluding that it was probably about the best that could be done. Even so, it did not come close to providing a system that a rational person might consider trustworthy.
Even if the engineering on the phone were perfect, a scanner brings within the user’s trust perimeter all those involved in targeting it – in deciding which photos go on the naughty list, or how to train any machine-learning models that riffle through your texts or watch your videos. Even if it starts out trained on images of child abuse that all agree are illegal, it’s easy for both insiders and outsiders to manipulate images to create both false negatives and false positives. The more we look at the detail, the less attractive such a system becomes. The measures required to limit the obvious abuses so constrain the design space that you end up with something that could not be very effective as a policing tool; and if the European institutions were to mandate its use – and there have already been some legislative skirmishes – they would open up their citizens to quite a range of avoidable harms. And that’s before you stop to remember that the European Court of Justice struck down the Data Retention Directive on the grounds that such bulk surveillance, without warrant or suspicion, was a grossly disproportionate infringement on privacy, even in the fight against terrorism. A client-side scanning mandate would invite the same fate.
But ‘if you build it, they will come’. If device vendors are compelled to install remote surveillance, the demands will start to roll in. Who could possibly be so cold-hearted as to argue against the system being extended to search for missing children? Then President Xi will want to know who has photos of the Dalai Lama, or of men standing in front of tanks; and copyright lawyers will get court orders blocking whatever they claim infringes their clients’ rights. Our phones, which have grown into extensions of our intimate private space, will be ours no more; they will be private no more; and we will all be less secure.
For a bit over a decade, SRI International and the University of Cambridge have been working to develop CHERI (Capability Hardware Enhanced RISC Instructions), a set of processor-architecture security extensions targeting vulnerability mitigation through memory safety and software compartmentalisation. In 2019, the UK’s Industrial Strategy Challenge Fund announced the £187M Digital Security by Design (DSbD) programme, which is supporting the creation Arm’s experimental CHERI-based Morello processor, System-on-Chip (SoC), and board shipping in early 2022, as well as dozens of industrial and academic projects to explore and develop CHERI-based software security. This week, UKRI will be launching an £8M funding call via EPSRC and InnovateUK to support UK-based academic and industrial CHERI/Morello software ecosystem development work. They are particularly interested in supporting work in the areas of OS and developer toolchain, libraries and packages, language runtimes, frameworks and middleware, and platform services on open-source operating systems — all key areas to expand the breadth and maturity of CHERI-enabled software. There is a virtual briefing event taking place on 5 October 2021, with proposals due on 8 December 2021.
The CHERI project at SRI International and the University of Cambridge are pleased to announce our second CHERI reference software-stack release. The release supports the CHERI-RISC-V and Arm Morello architectures. A complete set of development tools, including compiler, OS, debugger, and emulator are included in the release.
For this second release, the software has been packaged to be used in an easily accessible Docker image, hosted on Docker Hub. The focus of the release has been on CheriBSD, with the merging of pure capability kernels and Morello support to the development mainline. There are also many continued improvements to tooling, including updates to cheribuild (our unified build system), the CHERI Clang/LLVM compiler suite, the CHERI-extended GNU Debugger (gdb), and the QEMU full-system emulator (which now supports both the CHERI-RISC-V and Morello architectures).
The CHERI protection model provides architectural primitives to protect computer systems from widely-exploited security vulnerabilities. CHERI revises the hardware/software architectural interface with hardware support for capabilities that can be used for fine-grained memory protection and scalable software compartmentalization. Supported by DARPA (the US
Defense Advanced Research Projects agency) as well as UKRI (UK Research and Innovation) and its Digital Security by Design (DSbD) program, CHERI is the work of a large research team at the University of Cambridge, SRI International, Arm and many industrial and academic collaborators throughout the world.
The release along with full release notes are now available on line at https://cheri-dist.cl.cam.ac.uk/