Software Engineering Institute (SEI) Webcast Series: Recent Episodes
SEI Members of Technical Staff
Each webinar features an SEI researcher discussing their research on software and cybersecurity problems of considerable complexity. The webinar series is a way for the SEI to accomplish its core purpose of improving the state-of-the-art in software engineering and cybersecurity and transitioning this work to the community. The SEI is a federally funded research and development center sponsored by the U.S. Department of Defense and operated by Carnegie Mellon University. The SEI Webinar Series is produced by SEI Communications Outreach.
Many organizations are discovering, as they accelerate adoption of artificial intelligence (AI), that business and operational success with AI depends on far more than deploying AI models or experimenting with generative AI tools. Successful AI adoption occurs at the intersection of software engineering practices, the realities of system and enterprise architecture modernization, governance, cybersecurity, workforce readiness, workflow reengineering, operational integration, and enterprise strategy. Organizations must manage technological challenges that have intensified with AI adoption, including growing dependencies, vendor lock-in, and the imperative to innovate and scale quickly. Leaders must also adapt to new emerging realities, from the operational and financial demands of supporting multiple frontier models to the novel security and governance risks introduced by agentic AI approaches. Traditional approaches to technology transformation are no longer sufficient to thrive in this environment.
To address these emerging complexities and drive success, Carnegie Mellon University's Software Engineering Institute (SEI) collaborated with Accenture to develop the AI Adoption Maturity Model—an evidence-backed, field-tested instrument that provides a structured, yet agile, pathway for scaling AI capabilities across enterprises to ensure value and return on investment (ROI). This approach is designed for today's realities, including fast-paced technological change, limited time and resources, and the need for lightweight, actionable methods rather than burdensome documentation.
In this webcast, experts from the SEI and Accenture share technical insights and lessons learned from maturing AI adoption in complex environments. They will demonstrate how a nimble assessment instrument such as the road-tested AI Adoption Maturity Model fills critical gaps faced by organizations adopting AI.
What Will Attendees Learn?
• How AI maturity extends beyond isolated experimentation to encompass scalable, repeatable, measurable, and governed organizational capabilities.
• How common pitfalls and strengths that we observed in early adopter organizations during AI Adoption Maturity Assessments can influence AI adoption.
• How certain approaches to integrating existing risk management routines and security processes can support AI adoption.
• How an agile, lightweight maturity assessment approach can enable organizations to rapidly prioritize activities, align efforts, and make targeted progress.
AI is rapidly changing how software is produced but not the need to engineer software to meet business and mission goals. AI is enabling developers to move from idea to implementation at incredible speeds. However, this fast pace has implications that teams must manage. Product quality does not come for free, and there is some tendency for AI to accelerate the accumulation of technical debt. In addition, what works well on small code bases doesn't always work as well on large code bases. So, what's a good software engineer to do? Software engineering principles and practices are essential in guiding software engineers to use AI to achieve production-ready outcomes. In this webcast, experienced software engineers discuss their observations and lessons from applying AI-native software engineering and studying its use across multiple projects.
What Will Attendees Learn?
• Distinguish between "vibe coding" and software engineering
• Understand how software engineering principles improve the use of AI and where these principles need to be adapted to be used with AI
• Recognize the different criteria that can be used to assess the benefits of AI-native software engineering (e.g., productivity and software quality) and their potential tradeoffs
Metrics are about more than just compliance. They ground you in fact, enable informed decision making, and help your team understand performance while providing insight into program health. By understanding the why, what, and how behind metrics, your team can move from "just collecting data" to using metrics that enable better decision making, improve program outcomes, and deliver mission value. Get ready to hear all about our new Tactical Guide called Pack Light, Measure Right and Supplement called Decoding SWP Metrics.
What Will Attendees Learn?
• How to select a minimal but sufficient set of measures to get the insight you need
• How to implement measurement approaches that are compatible with iterative delivery
• The hidden value in the metrics that you're already collecting if you are on the Software Acquisition Pathway
Artificial intelligence (AI) systems offer tremendous potential, but compared to traditional software, they introduce novel safety and security risks. System theory provides a powerful lens for understanding these risks and developing effective mitigations. In this webcast, we'll introduce System Theoretic Process Analysis (STPA), a system-theory-based approach to safety analysis. We'll explain how STPA helps organizations build stronger assurances about the safety and security of complex systems, including those that incorporate AI.
What Will Attendees Learn?
• How complex systems fail due to design flaws and unsafe interactions—not just component failures
• How these types of accidents can occur in AI-enabled systems
• How to apply a system-theoretic perspective, including System Theoretic Process Analysis (STPA), to analyzing AI systems
• Practical insights into improving the design, testing, and operational use of AI systems to strengthen safety and security
Building an effective security operations capability such as a security operations center (SOC) has always been a challenging endeavor. Balancing the need to successfully integrate the people, processes, and technologies required to support your mission requires a deep understanding of your network and your threat landscape. New AI tools and technologies present both challenges and opportunities. These tools can complicate your operational environment but might be used by malicious actors to enhance their attacks. But AI can also be leveraged to build out and enable your SOC by covering gaps in tools, your workforce, and automation.
What Will Attendees Learn?
• How AI has changed the way SOC teams operate
• Ways to deploy AI tools in security operations settings
• How the SEI is using structured knowledge to build more effective SOCs
Are you confused about what should be in your Software Acquisition Pathway program's Capability Needs Statement (CNS)? You are not alone! One of the most frequent requests we receive is for a sample CNS. If only it was that easy! Never fear; our recently released Tactical Guide called Cracking the CNS Code will provide you with the practical insight you need to develop a CNS that will be effective for your program. We'll also talk about the Cracking the CNS Code's complementary supplement called How Does a CNS Drive the Process for Creating Working Software?
What Will Attendees Learn?
• What a CNS is
• Strategies for and approaches to creating a CNS
• How to break down your approved CNS into backlog items
Software management is too frequently ignored or addressed piecemeal in systems. Cyber threat actors take advantage of gaps and errors in their attacks, which they can accomplish throughout the lifecycle. Exploiting these gaps and errors allows them to compromise processes, practices, and procedures that touch a system's design, component development, and supply chain to bypass controls and leverage available vulnerabilities. Key software assurance activities must be embedded within the acquisition lifecycle to effectively combat these threat actors.
What Will Attendees Learn?
- How software assurance can be addressed with limited cost and schedule impact if it is effectively integrated into the acquisition lifecycle
- Which knowledge and resources are critical to software assurance and the risks that can be missed if they are underrepresented
- Key aspects of managing acquisition and development that are critical to software assurance and why they are important
AI is transforming both the threat landscape and our defensive capabilities. What does cyber mission readiness mean in this new environment? A researcher from the CERT Division of Carnegie Mellon University's (CMU's) Software Engineering Institute (SEI) describes the current challenges and emerging solutions that individuals and teams can use to build and sustain their cyber and AI-readiness. The webcast showcases Crucible, an open source framework that integrates learning management, hands-on labs, team exercises, competitions, threat sharing, and AI models into a unified cyber readiness platform.
What Will Attendees Learn?
- How to leverage AI to build cyber capacity and effectively share threat information across organizations
- Techniques for motivating performance through cyber challenge competitions
- Methods for using AI to assist in developing cyber-skills mastery via guided self-assessment
- AI-driven strategies for forging elite teams that are prepared for cyber and cognitive operations amid nation-state competitions
Is your program really ready to adopt the SWP? Next in the Software Acquisition Go Bag series, we'll walk you through our new Tactical Guide called SWP: Ready, Set, Go! This guide provides proven techniques to assess the project's readiness to adopt the SWP; identify any shortfalls; and obtain the resources, information, and support needed for success.
What Will Attendees Learn?
• How to know if your acquisition program is really ready to adopt the SWP
• What "Instill an Agile Culture" actually means for your program
• How programs enter or transition to the SWP
• A dispelled myth about adopting SWP
Many organizations practicing DevSecOps have collapsed under the weight of their own tooling. These organizations tried to solve delivery problems by stacking Kubernetes, Helm, GitOps controllers, scanners, and templating systems until no one can explain their own deployment path. This webcast cuts through that complexity and shows how right-sizing DevSecOps—reproducible environments, deterministic builds, type-safe configuration, and small iterative releases—restores velocity and reliability. We focus on what high-stakes teams actually need, not what vendors or compliance frameworks prescribe.
What Will Attendees Learn?
• Why complexity, not capability, is the primary barrier to fast, secure delivery
• How reproducibility, pre-rendering, and type-safe configuration eliminate entire categories of deployment failure
• How to design a "paved road" that scales across teams without drowning them in tooling
• How to move from episodic, end-state-driven integration to continuous, incremental delivery
This webcast offers a solution to the problem of poorly defined requirements in system design that can lead to software flaws, cost and time overruns, and stakeholder dissatisfaction. We will tell you how to use a structured process called the ATAM (architectural tradeoffs analysis method) to develop a system design by eliciting requirements, scenarios, and priorities from stakeholders. Then, we will explain how to measure compliance with those requirements during testing using DevSecOps principles and tools, such as the SEI's Silent Sentinel.
What Will Attendees Learn?
- What software quality attributes are and why they are important
- How to prioritize competing requirements
- How to ensure architectural requirements are satisfied during development in a DevSecOps toolchain
The SEI contributed its expertise to the development of the Software Acquisition Pathway (SWP), which the Department of Defense (DoD) issued in 2020 as DoD Instruction 5000.87. Since the SWP's issuance, SEI researchers have collaborated with DoD program teams and policy owners to effectively implement the pathway in different program contexts, identify barriers and challenges, and monitor outcomes. Throughout that work, we've identified common questions and stumbling blocks that programs encounter as they adopt the SWP. Answering these questions often warrants additional tools and resources that enable programs to position their SWP programs for success. With that in mind, the SEI announces the launch of the Software Acquisition Go Bag. Our SEI team has helped hundreds of DoD programs deliver software-enabled capability through our unique integration of data-driven insights, software engineering research, and acquisition science. We're "packing" that experience into Go Bag kits so program teams can implement proven practices.
Experts agree that quantum computing will likely become powerful enough to break modern-day encryption within the next 10–15 years on "Q Day." Once encryption is defeated, the computing world will never be the same. Organizations need to identify the correct courses of action to take today so that the sudden onset of quantum computing does not threaten their critical assets. In this webcast, Brett Tucker, Dan Justice, and Matthew Butkovic discuss the challenges expected with the realization of quantum computing capabilities. Furthermore, the group will provide possible responses to mitigate future impacts from the onset of quantum computing.
What Will Attendees Learn?
- what Q Day is
- events that may occur leading up to the actual day itself
- challenges that must be overcome in quantum computing (attendees need to learn and appreciate the factors that may influence the successful emergence of quantum computing)
- actions that can be taken today to build resilience from this technological disruption
Finding and fixing weaknesses and vulnerabilities in source code has been an ongoing challenge. There is a lot of excitement about the ability of large language models (LLMs, e.g., GenAI) to produce and evaluate programs. One question related to this ability is: Do these systems help in practice? We ran experiments with various LLMs to see if they could correctly identify problems with source code or determine that there were no problems. This webcast will provide background on our methods and a summary of our results.
What Will Attendees Learn?
• how well LLMs can evaluate source code
• evolution of capability as new LLMs are released
• how to address potential gaps in capability
Often, agile implementations are a struggle. Dedicated agile teams focus hard and deliver value on a regular cadence. But when results are tallied, the value teams produce may not fit neatly into the expectations of senior stakeholders. Why? In this webcast, Peter Capell addresses the importance of a practical vision to express outcomes, so that the program's “target picture” is clear to all parties involved. Peter highlights the value of tools such as Model-Based Systems Engineering (MBSE) in engineering processes, and how the combination of architecture and MBSE can anchor the implementation within those expectations.
What Attendees Will Learn:
• The concept of “just enough”
• How speed of delivery is only relevant when delivery is on target
• How the “targets” for team success begin with the program vision
• How modeling and architecture can serve as valuable tools to accomplish a practical vision
Finding and growing AI and Data talent is essential for mission success, but many skilled workers remain unseen because they lack traditional credentials. This session introduces practical strategies and prototype tools that help individuals demonstrate what they know while helping managers identify and evaluate emerging talent in these fields. Attendees will explore micro-assessments reflecting real data science and AI workflows, see how skills can be measured meaningfully at scale, and gain insights on fostering AI and Data readiness across the federal workforce. Whether you’re building your career or building your team, come learn how to connect talent with opportunity in the evolving AI landscape.
What Attendees Will Learn:
• Common barriers to finding and recognizing hidden AI and Data talent.
• The role of a practical work role rubric in aligning skills with mission needs.
• How prototype assessments and discovery tools can help surface and showcase talent.
Threat modeling is intended to help defend a system from attack. It tops the list of techniques recommended by the National Institute of Standards and Technology (NIST) to secure critical systems. In a world where people with malicious intent have deadlier tools at their disposal, defenders need to take advantage of Model-Based Systems Engineering (MBSE) to form mitigation strategies effective from early in the systems engineering lifecycle. This webcast will preview a workshop to be held during the 2025 Secure Software by Design conference to be held on August 19 and 20.
What Attendees Will Learn:
- How MBSE can aid cybersecurity analysis and design
- The value of MBSE for cyber threat modeling
- An overview of threat modeling techniques using MBSE
DevSecOps generates a lot of data valuable for better decision making. However, decision makers may not see all they need to in order to make best use of the data for continuous improvement. The SEI open source Polar tool unlocks the data, giving DevSecOps teams greater capability to automate, which in turn means they can innovate rapidly – without lessening quality or reducing security.
What Attendees Will Learn:
- Issues from complex DevSecOps pipelines
- What observability adds for DevSecOps efforts
- The way in which a new open-source tool, Polar, helps
Organizations looking to build and adopt artificial intelligence (AI)–enabled systems face the challenge of identifying the right capabilities and tools to support Machine Learning Operations (MLOps) pipelines. Navigating the wide range of available tools can be especially difficult for organizations new to AI or those that have not yet deployed systems at scale. This webcast introduces the MLOps Tool Evaluation Rubric, designed to help acquisition teams pinpoint organizational priorities for MLOps tooling, customize rubrics to evaluate those key capabilities, and ultimately select tools that will effectively support ML developers and systems throughout the entire lifecycle, from exploratory data analysis to model deployment and monitoring. This webcast will walk viewers through the rubric’s design and content, share lessons learned from applying the rubric in practice, and conclude with a brief demo.
What Attendees Will Learn:
• How to identify and prioritize key capabilities for MLOps tooling within their organizations
• How to customize and apply the MLOps Tool Evaluation Rubric to evaluate potential tools effectively
• Best practices and lessons learned from real-world use of the rubric in AI projects
DevSecOps practices foster collaboration among software development, security, and operations teams to build, test, and release software quickly and reliably. A high-stakes, high-security environment has challenged the implementation of these practices within the Department of Defense (DoD). The DoD Chief Information Officer (CIO) organization partnered with the Software Engineering Institute (SEI) to conduct the first study to baseline the state of DoD DevSecOps, highlight successes, and offer insights for next steps. George Lamb, DoD’s Director of Cloud and Software Modernization, joins the SEI team to discuss key results and how they will help the DoD ensure that its software ecosystem is effective, scalable, and adaptable to meet the challenges of today and tomorrow.
What Attendees Will Learn:
- Highlights from important success stories in DoD’s DevSecOps journey
- How the DoD is harvesting grassroot successes by individual software organizations to implement those successes at scale
- Keys to using data and building effective measurement strategies to enable optimization of software delivery
Did you know there are 500 million tweets per day? 3 billion monthly active Facebook users? 1 billion LinkedIn members? Are you one of them? In this webcast, Destiney Marie Plaza reveals how a hacker can use seemingly benign public information to customize an attack on a victim by showing a scenario-based attack and demo (using free and open-source tools). Additionally, you will learn how hackers can gather information about you, common mistakes that put your information at risk, and how to protect yourself.
What Attendees Will Learn:
- how to use open-source tools used to crack passwords, along with a methodology for how hackers may gain access to your accounts
- what makes a strong password and how such passwords can stave off automated cracking tools
- how a hacker sees you, so that you can take appropriate steps to protect yourself
Today, we have seen our national security organizations working to adopt modern software practices, particularly Agile methods and DevSecOps practices, efforts challenged by a mismatch of tempos between operational needs and development processes. The newly mandated Software Acquisition Pathway helps to align those tempos. However, to sustain a competitive advantage through software, we need to see our defense organizations recall and reapply disciplined engineering practices.
What Attendees Will Learn:
- An assessment of current efforts to adopt modern software practices
- Why and where the pace of adoption faces challenges
- Characteristics of the needed new level of performance
An organization’s cyber risk management practices must be rooted in organizational goals to be truly effective. In this webcast, Matt Butkovic, Greg Crabbe and Beth-Anne Bygum explore how best to align business and resilience objectives.
Why aren't malware analysis practitioners making more use academic research results? In this webcast, we suggest that one reason is the general difficulty of replicating and reproducing research results in this field. We randomly selected 100 papers on "malware classification" from Google Scholar results and attempted to replicate each one. We were only able to find released code for 6 of these 100 papers, and what's worse, only 6 of the 88 remaining papers contained a specific listing for the algorithm. We offer suggestions for improving the state of the field and end with a call to action for researchers to improve their methods so that their work will be useful for everyone.
What Attendees Will Learn:
- Why replication and reproduction are important topics in malware research
- The problems with data in malware research and suggestions to fix them
- The basics of using science in the field
The Defense Industrial Base (DIB) is a core element of the national security ecosystem. This point of intersection between private industry and the Department of Defense is a perpetual target for the Nation’s adversaries. In this Intersect, Matthew Butkovic and John Haller explore the development, and implementation, of the Cyber Maturity Model Certification (CMMC) as a means to better protect the DIB.
When it comes to recognizing threats, cybersecurity professionals may become distracted by big promises or ignore some obvious inspections. New claims made by the latest and greatest new apps draw attention away from network situational awareness best practices—like a dog distracted when it spots a squirrel. We also may deviate from making routine inspections that point toward further investigation—overlooking obvious needs right under our noses. Either becoming distracted or missing obvious inspections can cause us not to detect threats.
What Attendees Will Learn:
• The distinction between anomalies and threats
• Steps to analyze data to detect a threat
• The benefits of completing work on one threat
Can a cybersecurity parametric cost estimation model be developed? Every Department of Defense (DoD) program needs to account for, credibly estimate, budget/plan for, and assess the performance of its cybersecurity activities. Creating a cybersecurity parametric model would allow DoD programs to reliably estimate the effort and cost of cybersecurity activities, estimate an overall cybersecurity cost for a program, and obtain a defined and normalized set of cybersecurity data. In this webcast, Christopher Miller shares insights from a Carnegie Mellon University Software Engineering Institute study on cybersecurity cost estimating that can help national security organizations successfully deploy parametric cost modeling.
What Attendees Will Learn:
• a proposed work breakdown structure identifying cybersecurity-related activities and cost items, and existing descriptions of secure coding practices and levels of rigor for those practices based on data availability
• an approach to develop a cybersecurity parametric cost model
• a methodology to develop the cost model
Communications, both in times of crisis and during normal operations, are essential to the overall success and sustainability of an incident response or security operations team. How you plan for and manage these communications and how they are received and actioned by your audience will influence your trustworthiness, reputation, and ultimately your ability to perform incident management services effectively. This webcast leverages the National Institute of Standards and Technology (NIST) Cybersecurity Framework and the Forum of Incident Response and Security Teams (FIRST) CSIRT Services Framework to present communications responsibilities as part of both the standard incident management lifecycle and as an integral piece of crisis management support.
What Attendees Will Learn:
• various communication types or mechanisms for normal and crisis situations
• foundational aspects of managing communications with constituents, the public, and the media
• building blocks for an effective communications plan
Surviving disruptive cyber events requires a specific form of planning. One must strike a balance between defending against threats (e.g., managing conditions) and effectively handling the effects of disruption (e.g., managing consequences). Employing a model (such as the CERT Resilience Management Model) provides a catalog of practices and a system of measurement. Focusing on key attributes of performance permits a level of prediction not possible with a basic checklist. In this webcast, Greg Crabbe and Matt Butkovic share their experiences in establishing and maintaining operational resilience programs.
What Attendees Will Learn:
• how to link mission outcome with asset resilience
• how managing for security differs from managing for resilience
• how to apply a capability maturity model to the challenge
• how to begin analyzing requirements and constructing an operational resilience management program
Chief Information Security Officers (CISOs) perpetually navigate a dynamic set of challenges. Applying focus and aligning resources is imperative for success. In this Intersect, Matthew Butkovic and Gregory Touhill, reflect on 2024 and explore the topics that should be front of mind for CISOs in 2025. They provide insights and advice for those contemplating cybersecurity priorities.
No organization can comprehensively avoid disruptive cyber events. All must strive to maintain operational resilience during times of organizational stress. Ransomware incidents create disruption that can be fatal to the unprepared. In this webcast, we explore how to maintain operational resilience during a ransomware incident. Experts with varied backgrounds provide practical advice for improving your resilience and survivability.
What attendees will learn:
• best practices for ransomware response
• moving beyond security and planning for resilience
• pitfalls to avoid in the planning and response processes
As the strategic importance of AI increases, so too does the importance of defending those AI systems. To understand AI defense, it is necessary to understand AI offense—that is, counter AI. In this session, Matthew Butkovic, CISA, CISSP, technical director for risk and resilience, and Nathan VanHoudnos, senior machine learning researcher explore the fundamentals of counter AI.
Health-care organizations are seemingly besieged by a complex set of cyber threats. The consequences of disruptive cyber events in health care are in many ways uniquely troubling. Health-care organizations often face these challenges with modest resources. In this webcast, Matthew Butkovic and Darrell Keeling will explore approaches to maximize return on cybersecurity investment in the health-care context. This will include applying fundamental measures of operational resilience.
What Attendees Will Learn:
- How to yield maximum return on cybersecurity investment in health care
- How to shift thinking from cybersecurity to operational resilience
- How to employ free or low-cost cybersecurity resources in the health-care context
Traditionally, independent verification and validation (IV&V) is performed by an independent team throughout a program’s milestones or once the software is formally delivered. This approach allows the IV&V team to provide input at the various milestone gates. As more programs move to an Agile approach, those milestones aren’t as clearly defined since requirements, design, implementation, and testing all happen iteratively, sometimes over years of development. In this new paradigm, IV&V teams are struggling to figure out how to add value to the program earlier in the lifecycle by getting in phase with development. This webcast will highlight a novel approach to providing IV&V for projects using an Agile or iterative software development.
What Attendees Will Learn:
- What adopting an Agile mindset for IV&V could look like
- How focusing on capabilities and using a risk-based perspective could help drive planning for your team
- Techniques to help the IV&V team get more in phase with the developer while remaining independent
In an era where digital transformation is paramount, the potential of generative artificial intelligence (AI) extends far beyond automation. In this webcast, Tyler Brooks, Shannon Gallagher, and Dominic Ross aim to demystify AI and illustrate its transformative power in achieving scalability, adapting to changing landscapes, and driving digital innovation. The speakers delve into the practical applications of generative text, imagery, and synthetic audio, showcasing how these technologies can revolutionize various workflows.
What Attendees Will Learn:
- Practical applications of generative text, imagery, and synthetic audio
- Impact on the scalability of educational content delivery
- How synthetic audio is transforming AI education
Within a very short amount of time, the productivity and creativity improvements envisioned by generative artificial intelligence (AI), such as using tools based on large language models (LLMs), have taken the software engineering community by storm. The industry is in a race to develop your next best software development tool. Organizations are perplexed by trying to find the right balance between staying ahead in the race and protecting their data and systems from potential risks presented by using generative AI as part of their software development tool chain. There are haters, evangelists, and everything in between. Software engineering education and educators have a special role. No matter how they perceive the opportunities and challenges of generative AI approaches, software engineering educators are going through a watershed moment that will change how they educate the next generation of software engineers. In this webcast, three experts in software engineering will discuss how generative AI is influencing software engineering education and how to balance key skills development with incorporating generative AI into software engineering curricula.
What Attendees Will Learn:
• how software engineering education is challenged by the increasing popularity of generative AI tools
• how software engineering educators can take advantage of generative AI tools
• what fundamental skills will be critical to teach to software engineering students in the era of generative AI
Traditionally, cybersecurity has focused on finding and removing vulnerabilities. This is like driving backward down the highway using your rearview mirror. Most breaches are due to defects in design or code; thus, the only way to truly address the issue is to design and build more secure solutions. In this webcast, Tim Chick discusses how security is an integral aspect of the entire software lifecycle as a result of following deliberate engineering practices focused on reducing security risks through the use of software assurance techniques.
What Attendees Will Learn:
• The importance of cybersecurity and examples of when security has failed
• Qualities to look at when evaluating third-party software
• The relationship between quality and security
• Engineering techniques used throughout the development lifecycle to reduce cyber risks
Modern analytic methods, including artificial intelligence (AI) and machine learning (ML) classifiers, depend on correlations; however, such approaches fail to account for confounding in the data, which prevents accurate modeling of cause and effect and often leads to prediction bias. The Software Engineering Institute (SEI) has developed a new AI Robustness (AIR) tool that allows users to gauge AI and ML classifier performance with unprecedented confidence. This project is sponsored by the Office of the Under Secretary of Defense for Research and Engineering to transition use of our AIR tool to AI users across the Department of Defense. During the webcast, the research team will hold a panel discussion on the AIR tool and discuss opportunities for collaboration. Our team efforts focus strongly on transition and provide guidance, training, and software that put our transition collaborators on a path to successful adoption of this technology to meet their AI/ML evaluation needs.
What Attendees Will Learn:
• How AIR adds analytical capability that didn’t previously exist, enabling an analysis to characterize and measure the overall accuracy of the AI as the underlying environment changes
• Examples of the AIR process and results from causal discovery to causal identification to causal inference • Opportunities for partnership and collaboration
There is a lot of documentation about a zero trust architecture, as well as directives that it be used for U.S. federal agencies and the Department of Defense (DoD), but little information on how to go about implementing it to improve an organization’s enterprise or DoD weapon system security. Use cases typically describe requirements for these systems, but they do not provide the contextual awareness that organizations need to help them create a prioritized roadmap to implement zero trust. In this webcast, Tim Morrow, Rhonda Brown, and Elias Miller discuss an approach that organizations can use to help develop the contextual awareness needed to apply a zero trust strategy.
What Attendees Will Learn:
- Overview of a zero trust strategy
- Roadmap focusing on zero trust for the DoD
- Engineering approach for mission/workflow
- Use of a scenario to help reason about zero trust considerations
- Awareness of an upcoming SEI Zero Trust Industry Day event
According to the Verizon Data Breach Report, Log4j-related exploits have occurred less frequently over the past year. However, this Common Vulnerabilities and Exposures (CVE) flaw was originally documented in 2021. The threat still exists despite increased awareness. Over the past few years, the Software Engineering Institute (SEI) has developed guidance and practices to help organizations reduce threats to U.S. supply chains. In this webcast, Brett Tucker and Matthew Butkovic, answer your enterprise risk management questions to help your organization achieve operational resilience in the cyber supply chain.
What attendees will learn:
- Enterprise risk governance and how to assess organization’s risk appetite and policy as it relates to and integrates cyber risks into a global risk portfolio
- Regulatory directives on third-party risk
- The agenda and topics to be covered in the upcoming CERT Cyber Supply Chain Risk Management Symposium in February
We stand at a pivotal moment in software engineering, with artificial intelligence (AI) playing a crucial role in driving approaches poised to enhance software acquisition, analysis, verification, and automation. While generative AI tools initially sparked excitement for their potential to reduce errors, scale changes effortlessly, and drive innovation, concerns have emerged. These concerns encompass security risks, unforeseen failures, and issues of trust. Empirical research on generative AI development assistants reveals that productivity and quality gains depend not only on the sophistication of tools but also on task flow redesign and expert judgment.
In this webcast, Software Engineering Institute (SEI) researchers will explore the future of software engineering and acquisition using generative AI technologies. They’ll examine current applications, envision future possibilities, identify research gaps, and discuss the critical skill sets that software engineers and stakeholders need to effectively and responsibly harness generative AI’s potential. Fostering a deeper understanding of AI’s role in software engineering and acquisition accentuates its potential and mitigates its risks.
What Attendees Will Learn
• how to identify suitable use cases when starting out with generative AI technology
• the practical applications of generative AI in software engineering and acquisition
• how developers and decision makers can harness generative AI technology
Compliance standards, privileged access management, software bills of materials (SBOMs), maturity models, cloud services, vulnerability management, etc. The list of potential solutions to supply chain risk management (SCRM) challenges seems unending as much as it is daunting to address. In this webcast, Brett Tucker explores some of these solutions. More importantly, he renews an emphasis on using robust enterprise risk management to achieve operational resilience in the cyber supply chain.
What attendees will learn
- A means of decomposing strategic objectives and critical services into high-value assets that point to prioritization of limited risk response resources
- Enterprise risk governance, appetite, and policy as they relate to and integrate cyber risks into a global risk portfolio
- The application and impacts of Cybersecurity Maturity Model Certification (CMMC) and other regulatory directives on third-party risk
- A kick-off announcement about the SEI CERT Supply Chain Risk Management Symposium to be held in February 2024
Generative AI (GenAI) has been around for decades, but the latest leap in progress, fueled by high-capability large language models (LLMs), image and video generators, and AI pair programmers, has captivated audiences across a variety of disciplines. What can GenAI do well? What are the risks and opportunities of using GenAI?
SEI experts Doug Schmidt, Rachel Dzombak, Jasmine Ratchford, Matt Walsh, John Robert and Shing-hon Lau conducted a live question-and-answer session driven by the audience.
Here’s what attendees will learn:
- The risks and rewards of generative AI
- The future of LLMs
- SEI research in this area
AI system trustworthiness is dependent on end users’ confidence in the system’s ability to augment their needs. This confidence is gained through evidence of the system’s capabilities. Trustworthy systems are designed with an understanding of the context of use and careful attention to end-user needs. In this webcast, SEI researchers discuss how to evaluate trustworthiness of AI systems given their dynamic nature and the challenges of managing ongoing responsibility for maintaining trustworthiness.
What attendees will learn:
- Basic understanding of what makes AI systems trustworthy
- How to evaluate system outputs and confidence
- How to evaluate trustworthiness to end users (and affected people/communities)
A Software Bill of Materials (SBOM) is a comprehensive list of software components involved in the development of a software product. While recently gaining attention in the context of security, SBOMs have limited value unless properly integrated into effective cyber risk management processes and practices. The SEI SBOM Framework compiles a set of leading practices for building an SBOM and using it to support risk reduction.
The SEI SBOM Framework provides a roadmap for managing vulnerabilities and risks in third-party software, including commercial-off-the-shelf (COTS) software, government-off-the-shelf (GOTS) software, and open-source software (OSS). A set of use cases informed the identification of SBOM practices, including building an SBOM and using it to manage risks to software intensive systems. These foundational practices were augmented using key security management concepts, such as the need to address requirements, planning and preparation, infrastructure, and organizational support. In this webcast, Charles Wallen, Carol Woody, and Michael Bandor discuss how organizations can connect SBOMs to acquisition and development to support improved system and software assurance.
Insider threats pose an enduring, ever-evolving risk to an organization’s critical assets that require enterprise-wide participation to manage effectively. Many organizations struggle to make critical tasks in insider risk management “stick,” relying on several crutches to drive temporary organizational change, only to see those changes come undone and have incidents slip through the cracks. In this webcast, we’ll discuss those crutches and identify themes of best practices observed over two decade of researching insider threat and building insider risk management programs that organizations can use to institutionalize key components of effective insider risk management.
What attendees will learn:
• How to identify drivers of change to an organization’s insider risk posture
• How to differentiate between one-time and routine activities in the planning and implementation of an insider risk management program
• How to measure the maturity of those routine activities
In this webcast, Fred Schenker, Jerome Hugues, and Linda Parker Gates discuss the benefits of using a model-based approach to improve the design of a CPS’ embedded computing resources. This is accomplished by (1) building virtual architectural models of the CPS’ embedded computing resources early in the system development lifecycle and (2) using these models to predict computing system constraints and component integration issues. They will discuss the cultural resistance to adopting the model-based approach, and how established justification methods, e.g., Return on Investment, are being used to stifle the adoption. Finally, some alternatives to ROI will be proposed that would be more effective justification mechanisms.
The Rust programming language makes some strong claims about the security of Rust code. In this webcast, David Svoboda and Joe Sible will evaluate the Rust programming language from a cybersecurity perspective. They will examine Rust's security model, both in what it promises and its limitations. They will also examine how secure Rust code has been seen in practice and conclude with discussing the overall maturity and stability of the Rust ecosystem.
What attendees will learn:
- The Rust Security Model
- Limitations of the Rust Security Model
- Rust code in the current vulnerability ecosystem
- Rust code stability and maturity
Historically, a lot of discussion in software security focused on the project level, emphasizing code scanning, penetration testing, reactive approaches for incident response, and so on. Today, the discussion has shifted to the program level to align with business objectives. In the ideal outcome of such a shift, software teams would act in alignment with business goals, organizational risk, and solution architecture and would understand that security practices are integral to business success. However, the shift from project- to program-level thinking brings lots of challenges. In this webcast, Hasan Yasar and Joe Yankel discuss the top 5 challenges and barriers to implementing DevSecOps practices and describe some solutions for overcoming them.
What attendees will learn:
- The DevSecOps ecosystem and how it aligns with business objectives
- The DevSecOps challenges and barriers
- How to overcome the top 5 challenges
- Practical solutions for your business needs
- How your system architecture drives your DevSecOps ecosystem
Classic tool suites that are used to process network flow records deal with very limited detail on the network connections they summarize. These tools limit detail for several reasons: (1) to maintain long-baseline data, (2) to focus on security-indicative data fields, and (3) to support data collection across large or complex infrastructures. However, a consequence of this limited detail is that analysis results based on this data provide information about indications of behavior rather than information that accurately identifies behavior with high confidence. In this webcast, Tim Shimeall and Katherine Prevost discuss how to use IPFIX-formatted data with detail derived from deep packet inspection (DPI) to provide increased confidence in identifying behavior.
In this webcast, Tom Scanlon, Matthew Walsh and Jeffrey Mellon discuss approaches to using data science and machine learning to address cybersecurity challenges. They provide an overview of data science, including a discussion of what constitutes a good problem to solve with data science. They also discuss applying data science to cybersecurity challenges, highlighting specific challenges such as detecting advanced persistent threats (APTs), assessing risk and trust, determining the authenticity of digital content, and detecting deepfakes.
What attendees will learn:
- Basics of data science and what makes for a good data science problem
- How data science techniques can be applied to cybersecurity
- Ways to get started using data science to address cybersecurity challenges
As Artificial Intelligence permeates mission-critical capabilities, it is paramount to design modular solutions to ensure rapid evolution and interoperability. During this webcast, we’ll discuss some of the primary quality attributes guiding such design, and how a Next Generation Architecture can facilitate an integrated future state.
What attendees will learn:
- current challenges facing AI engineering
- approaches to promoting interoperability across AI solutions
- considerations for facilitating modularity and reuse in design
All technology acquired by an organization requires the support of (or integration with) components, tools, and services delivered by a diverse set of supply chains. However, the practices critical to addressing supply chain risks are typically scattered across many parts of the acquiring organization, and they are performed in isolated stovepipes. This situation causes inconsistencies, gaps, and slow response to crises. The Acquisition Security Framework (ASF) addresses this problem by combining leading cyber practices that help organizations manage supply chain risk and define the collaborations critical to securely acquiring, engineering, and operating software-reliant systems. The goals, practices, and processes that structure the ASF have been demonstrated as effective for managing risk and improving resilience. The ASF is consistent with published guidelines for supply chain risk management from ISO, NIST, and DHS.
What attendees will learn:
This webcast will introduce attendees to the ASF and demonstrate the ways in which the ASF provides a roadmap to help organizations build security and resilience into a system rather than “bolt on” these characteristics after deployment. The webcast will also examine how, following deployment, the ASF guides the ongoing management of system risk and resilience as the technology, threats, and requirements evolve over the system’s lifecycle.
ASF includes
- leading security and resilience practices critical to supply chain risk management
- a pathway for proactive process management that fosters effective collaboration across the range of stakeholders responsible for acquiring, developing, and deploying software-reliant systems
Understanding and articulating cybersecurity risk is hard. With the adoption of DevSecOps tools and techniques and the increased coupling between the product being built and the tools used to build them, the attack surface of the product continues to grow by incorporating segments of the development environment. Thus, many enterprises are concerned that DevSecOps pipeline weaknesses can be abused to inject exploitable vulnerabilities into their products and services.
Using Model Based Systems Engineering (MBSE), a DevSecOps model can be built that considers system assurance and enables organizations to design and execute a fully integrated DevSecOps strategy in which stakeholder needs are addressed with cybersecurity in all aspects of the DevSecOps pipeline. An assurance case can be used to show the adequacy of the model for both the pipeline and the embedded or distributed system. While builders of embedded and distributed systems want to achieve the flexibility and speed expected when applying DevSecOps, reference material and a repeatable defensible process are needed to confirm that a given DevSecOps pipeline is implemented in a secure, safe, and sustainable way.
What Attendees will Learn:
- an approach to evaluate and mitigate the risk associated with attackers exploiting DevSecOps pipeline weaknesses and vulnerabilities
- how to structure an assurance case around the core capabilities of a DevSecOps pipeline
As a Software Engineering community, we started to hear new words with new definitions to achieve some challenges with deciding the shelf life of said terms. Some examples include: DevOps is dead, long live NoOps, SecOps, NoCode, SRE, GitOps, and recently Platform Engineering. We often confuse these terms in order to achieve certain software engineering job types. Then the organization decides to implement one or a combination of these terms and restructures the engineering team. However, it can often be cumbersome because many tech professionals are still unfamiliar with the technologies and “new buzzwords” are required to implement a complete SW delivery pipeline to meet the business needs. It is becoming very challenging to find the right way. We should all step back and ask ourselves “what is our why” to deliver new capabilities in a timely, affordable, and secure way. Let’s discuss how we can clear up this word puzzle and find our journey.
What Attendees will Learn:
• How to align your business objectives with your SW engineering practices?
• What is the science behind DevOps?
• Understand role vs responsibility
• How do I get started on implementing true DevOps?
• How to become an agile to overcame new obstacles?
Infrastructure as code (IaC) is a concept that enables organizations to automate the provisioning and configuration of their IT infrastructure. This concept also aids organizations in applying the DevOps process (plan, code, build, test, release, deploy, operate, monitor, repeat) to their infrastructure. Ansible is a popular choice within the IaC tool landscape for realizing this goal.
The lack of qualified cybersecurity professionals in the United States is a threat to our national security. We cannot adequately protect the systems that our government, economy, and critical infrastructure sectors rely on without an appropriately sized cyber workforce. By some estimates, there are over 700,000 cybersecurity job openings across the United States, with 39,000 of those in the public sector alone. Fortunately, the federal government recognizes that the cyber workforce needs to be strengthened and is implementing efforts to address this need at a national strategic level.
In this webcast, we will examine how to use principles and concepts from Agile development to help cyber workforce development initiatives remain adaptable and effective in the continuously evolving landscape of the cyber domain.
Ransomware poses an imminent threat to most organizations. Whereas most traditional cyber attacks require extended threat actor engagement to seeking out critical information, exporting data, and demanding ransom from victims, ransomware shortens the process and puts immediate pressure on the victim to respond with payment. Unfortunately, the rise of artificial intelligence (AI) and other novel attack techniques have made these attacks more ubiquitous as they are pernicious. In this talk, Brett Tucker will discuss a novel means for assessing an organization to determine its susceptibility to ransomware and explore the organization’s resilience to recover normal operations after a successful attack. Attendees will learn about the key domains for analysis and practical tips for facilitating resilience assessments.
By all recent measures, the cybersecurity workforce is woefully understaffed. According to (ISC)², the cyber workforce gap in the United States was 377,000 open positions in 2021. The Software Engineering Institute (SEI) at Carnegie Mellon University (CMU) has been working with the U.S. government to development novel approaches designed to shrink this gap. This talk will focus on open source initiatives that bring innovative ideas to cybersecurity modeling and simulation, assessment, and competitions
What Attendees will Learn:
- Understanding of the challenges facing the cyber workforce and how to build engaging content that will help close the workforce gap
- Where to find open source projects developed by the Software Engineering Institute to build cyber exercises, training labs and simulations
- Hands on experience that can be immediately applied to workforce development initiatives in their own organization
In this webcast, Carol Smith, Carrie Gardner, and Michael Mattarock discuss maturing artificial intelligence (AI) practices based on our current body of knowledge. Much as it did for software engineering in the 1980s, the SEI has begun formalizing the field of AI engineering, beginning with identifying three fundamental pillars to guide AI engineering: human-centered, scalable, and robust and secure. Watch to learn more about these pillars and how they can be used to help national defense and security agencies adopt and develop AI.
In this webcast, Carol Smith, Carrie Gardner, and Michael Mattarock discuss maturing artificial intelligence (AI) practices based on our current body of knowledge. Much as it did for software engineering in the 1980s, the SEI has begun formalizing the field of AI engineering, beginning with identifying three fundamental pillars to guide AI engineering: human-centered, scalable, and robust and secure. Watch to learn more about these pillars and how they can be used to help national defense and security agencies adopt and develop AI.
In this webcast, Shannon Gallagher and Dominic Ross discuss what deepfakes are, and how they are building AI/ML tech to distinguish real from fake. They will start with some well-known examples of deepfakes and discuss what makes them distinguishable as fake for people and computers.
Agile and DevSecOps have revolutionized software engineering practices. The strategies put forward in Agile and DevSecOps have eased many software engineering challenges and paved the way for continuous deployment pipelines. But what do you do when you're facing a problem that doesn't fit the model of a pure software engineering project?
In this webcast, we will share our experiences applying Agile and DevSecOps practices in atypical ways. We will focus our discussion around two atypical examples:
1) managing a program office’s acquisition process
2) developing and maintaining computing enclave operations
Using these examples, we will discuss the specific applications of the tools and practices we used to enhance a team's capabilities and better support end users' missions. We will also share the lessons we learned along the way.
Complex, cyber-physical DoD systems, such as aircraft, depend on correct timing to properly and reliably execute crucial sensing, computing, and actuation functions. In this webcast, SEI staff members Bjorn Andersson, PhD, Dionisio de Niz, PhD, and William Vance of the U.S. Army Combat Capabilities Development Command Aviation & Missile Center discuss using real-time software on multicore processors. Specifically, they review the challenges that DoD and civilian systems face and the proven solutions that are available.
The Forrester report, "The Definition of Modern Zero Trust," defines Zero Trust as an information security model that denies access to applications and data by default. Zero Trust adoption can be difficult for organizations to undertake. It is not a specific technology to adopt; instead, it’s an initiative that an enterprise must understand, interpret, and implement. In this webcast, SEI CERT Division Director Greg Touhill, and Dr. Chase Cunningham, chief strategy officer at Ericom Software, answered questions and discussed what you need to implement a Zero Trust strategy.
What You Will Learn
• Why Zero Trust is a strategy and not a technology
• Steps to implement Zero Trust
• Examples of past compromises and historical failures that were not only enabled by, but powered by, inherent trust
The status quo for how we acquire cyber-physical weapon systems (CPS) needs to be changed. It is almost certain (for any acquisition of a CPS) that there will be cost overruns, schedule delays, and/or the loss of promised warfighter capability. Improved product development technologies could be applied, but they have not been adopted widely. We will discuss the status quo, alternative approaches, and how to motivate the community of CPS acquirers and suppliers to improve.
What attendees will learn:
- Characteristics of the current status quo how CPS systems are acquired and developed.
- Principles of Value Engineering and how they can be applied to the acquisition and development of CPS.
- Approaches that acquirers and suppliers can take to improve the status quo.
In this episode, Grace Lewis and Shane McGraw discuss how the SEI is applying research, through its highly successful Tactical and AI-Enabled Systems (TAS) initiative, to develop foundational principles, innovative solutions, and best practices for architecting, developing, and deploying tactical and AI-enabled systems. These systems will provide solutions for teams operating in remote, tactical edge locations where computing resources are constrained.
Lewis and McGraw explain that the TAS initiative is seeking to add a software engineer to conduct research, build prototype technologies, and collaborate with military programs to introduce new technology. The results of this work will be seen in research papers that advance the state-of-the-art in software engineering and in the fielding of better tactical and AI-enabled systems.
We are hiring: Software Engineer https://cmu.wd5.myworkdayjobs.com/en-US/SEI/job/Pittsburgh-PA/Software-Engineer_2017511-1
In this webcast, Carol Woody presents the scope of a cybersecurity engineering strategy for DevSecOps along with the criticality of sharing information with direct and indirect stakeholders.
In this webcast, Brett Tucker, Ryan Zanin, and Abid Adam discuss the critical factors for risk executives to be successful to not only protect critical assets but also to take advantage of new opportunities created via the pandemic.
Zero Trust Architecture adoption is a challenge for many organizations. It isn't a specific technology to adopt; instead, it’s a security initiative that an enterprise must understand, interpret, and implement. Enterprise security initiatives are never simple, and their goal to improve the enterprise’s cybersecurity posture requires the alignment of multiple stakeholders, systems, acquisitions, and exponentially changing technology. This alignment is always a complex undertaking and requires cybersecurity strategy and engineering to succeed.
What attendees will learn:
• The purpose of a Zero Trust Architecture
• Zero Trust Architecture components
• How to think about Zero Trust Architecture transition
In its 2021 report, the National Security Commission on AI (NSCAI) wrote, "The impact of artificial intelligence (AI) on the world will extend far beyond narrow national security applications." How do we move beyond those narrow AI applications to gain strategic advantage? Join Dr. Matt Gaston, Director of the SEI AI Division, Dr. Steve Chien, NSCAI Commissioner and Technical Group Supervisor of the Artificial Intelligence Group and Senior Research Scientist in the Mission Planning and Execution Section at the Jet Propulsion Laboratory, California Institute of Technology, and Dr. Jane Pinelis, Chief of Test and Evaluation of AI/ML at the DoD Joint AI Center (JAIC) for a discussion on scaling AI. Carnegie Mellon University is proud to partner with NSCAI in this discussion, part of an ongoing series of virtual panel discussions to realize the future of AI.
What attendees will learn:
• NSCAI recommendations for scaling AI
• How AI Engineering can scale the impact of mission capabilities
• Where to find leading AI Engineering practices
• Challenges and opportunities for the future of AI
Self-driving cars are being tested in our cities, bespoke movie and product recommendations populate our apps, and we can count on our phones to route us around highway traffic... Why, then, do most AI deployments fail? What is needed to create, deploy, and maintain AI systems we can trust to meet our mission needs, particularly for defense and national security? The SEI recently launched an AI Division to ensure that our researchers are working to address these hard questions. In this question and answer session, Dr. Rachel Dzombak and Dr. Matt Gaston share their points of view on what AI engineering is today and where the field is going. Learn about building AI better with the nascent discipline of AI Engineering and how the SEI plans to leverage the new AI Division to advance human-centered, robust and secure, and scalable AI systems.
What attendees will learn:
• How to find AI Engineering lessons in your own AI practices
• What’s needed to build an AI Engineering mindset on your team
• Leading AI Engineering practices
• How to engage with a national initiative dedicated to advancing the discipline of AI Engineering
• How the SEI is growing our portfolio of work in the AI Division
Misuse of authorized access to an organization’s critical assets is a significant concern for organizations of all sizes, missions, and industries. We at the CERT National Insider Threat Center have been collecting and analyzing data on incidents involving malicious and unintentional insider since 2001, and have worked with numerous organizations across government, industry, and academia to develop and validate controls and best practices to address these concerns.
In this webcast, as a part of National Insider Threat Awareness Month, our experts provide an overview of the ongoing research in this area, and answer questions about how the threat landscape continues to evolve, and what organizations can and should do to address insider threats.
What Attendees Will Learn:
• The complexities of insider risk management and strategies for effectively balancing insider risk management program operations across the dimensions of people, organization, and management. • The latest findings from the CERT National Insider Threat Center’s research into the different types of insider incidents – motivations, vulnerabilities, and common attack paths
• The changing landscape of insider threat and a look into the future
• The newest best practices and other resource that are available through the CERT National Insider Threat Center
The software development lifecycle has changed a lot and continues to evolve. Almost every company now is a software company. Meeting business needs and adapting to the speed of the market for new features requires an agility mindset and continuous-delivery techniques throughout application-development lifecycles.
You have software development and deployment questions, such as: Where do I start? How do I establish good continuous integration/deployment practices? What about security?
Hasan has the answers! SEI’s Hasan Yasar hosts a software development question and answer session.
What attendees will learn:
• how DevSecOps and Agile are generating more and more questions in DoD environments
• where software development is heading
• continuous-delivery techniques throughout application-development lifecycles
• why constant interaction between developers and information security teams is needed throughout the entire SDLC
In a DevSecOps world the software supply chain extends beyond libraries upon which developed software depends. In this webinar we will look at the Solarwinds incident as a worst-case exemplifying the breadth of the software supply chain issues confronting complex DevSecOps programs. We will explore the important architectural aspects of DevSecOps that are impacted by the software supply chain that require attention and potential mitigations to detect and respond to potential incidents.
What attendees will learn:
• The software supply chain issue is broad and impacts multiple aspects of DevSecOps
• Programs need to be aware of how the software they leverage presents risks
• Mitigation strategies must be put in place to address potential issues at the architectural level
How do you teach cybersecurity to a middle school student? To a soldier? To some of the best hackers in the country? How do you evaluate all of these audiences’ skills? Cybersecurity training has been an ongoing challenge for decades. The key to making the best use of your training dollar is to craft training that matches your audience’s needs and engages them in a meaningful manner. When you create an experience so enthralling that your audience is logging in on nights and weekends just to continue participating, the value of immersive training truly shines. Join us during this webinar as Rotem Guttman shares the lessons he’s learned over a decade of developing engaging, immersive training and evaluation environments for a variety of audiences.
What attendees will learn:
• How to make cybersecurity training engaging
• What motivates different types of learners
• The history of enhanced cybersecurity training at the SEI
Managing third-party relationships, such as pubic cloud service providers, requires a set of skills often unfamiliar to many technologists. These relationships are constructed on a foundation of verifiable trust. This requires managing the cybersecurity performance of third parties via contractual mechanisms rather than the traditional line-of-sight practices used internal to an organization. Chief among these mechanisms are service-level agreements (SLAs). Cybersecurity SLAs are vital to the success of third-party relationships and a core component of sound governance.
What Attendees Will Learn
• How to design and implement meaningful SLAs
• How best to use SLAs to drive third-party cybersecurity performance
• The limits of SLAs as a third-party risk management tool
IEEE 2675 standard specifies technical principles and practices to build, package, and deploy systems and applications in a reliable and secure way. The standard focuses on establishing effective compliance and IT controls. It presents principles of DevOps including mission first, customer focus, shift-left, continuous everything, and systems thinking. It also describes how stakeholders, including developers and operations staff, can collaborate and communicate effectively. Co-authors will discuss their personal experience applying the principles and practices in organizations.
What attendees will learn:
• Learn DevOps for systems of systems
• What DevOps standards means
• How to read the DevOps standard and apply to your organization
• Key DevOps principles and practices
According to recent estimates, around 85% of AI projects fail to move from conceptualization to implementation. Why are these failures happening, and how can we prevent them? AI engineering is an emergent discipline focused on developing tools, systems, and processes to enable the application of artificial intelligence in real-world contexts. The SEI is leading the national initiative to create an AI engineering discipline to operationalize human-centered, robust and secure, and scalable AI.
Privacy protection isn't just a compliance activity. but It’s also a key area of organizational risk that requires enterprise-wide support and participation; careful planning; and forward-leaning, data-driven controls. In this webcast, we highlight best practices for privacy program planning and implementation. We present strategies for leveraging existing capabilities within your organization to further advance privacy program building, and look ahead to emerging research and operational needs for modernizing privacy programs.
What Attendees Will Learn?
• The state of the practice for privacy program planning and development
• How to align privacy program planning and development activities with related efforts within your organization
• Areas of ongoing and future research into privacy frameworks, privacy risk management, and privacy controls efficacy
There is some confusion about how the paradigms of DevOps and Digital Engineering fit together. In the case of software-intensive systems, we believe DevOps practices are an enabler for Digital Engineering, in many forms. During this webcast, we introduced the relatively new concept of Digital Engineering and how we believe DevOps actually complements/enables many of the goals of Digital Engineering.
What attendees will learn:
- What Digital Engineering is
- Who is using Digital Engineering
- How implementing DevOps can enable expansion into Digital Engineering
Speakers: Hasan Yasar and David Shepard
Many organizations struggle in applying DevSecOps practices and principles in a cybersecurity-constrained environment because programs lack a consistent basis for managing software intensive development, cybersecurity, and operations in a high-speed lifecycle. We will discuss how an authoritative reference, or Platform Independent Model (PIM), is needed to fully design and execute an integrated DevSecOps strategy in which all stakeholder needs are addressed, such as engineering security into all aspects of the DevSecOps pipeline to include both the pipeline and the deployed system.
We will discuss how a PIM of a DevSecOps system can be used to 1) Specify the DevSecOps requirements to the lead system integrators who need to develop a platform-specific solution that includes the system and CI/CD pipeline.
2) Assess and analyze alternative pipeline functionality and feature changes as the system evolves.
3) Apply DevSecOps methods to complex systems that do not follow well-established software architectural patterns used in industry.
4) Provide a basis for threat and attack surface analysis to build a cyber assurance case in order to demonstrate that the software system and DevSecOps pipeline are sufficiently free from vulnerabilities and function only as intended
The recent SolarWinds incident demonstrated the challenges of securing systems when they are the product of complex supply chains. Responding effectively to breaches and hacks requires a cross-section of technical skills and process insights. In this webcast, we explored the lifecycle of the SolarWinds activity and discussed both technical and risk assessment to prepare organizations to defend against this type of incident.
What attendees will learn:
*Technical details regarding the SolarWinds vulnerabilities and exploits
*Supply chain risk management principles required to reduce the risk of future incidents
*Advice on the core operational capabilities required to respond to and recover from the SolarWinds hack
Speakers: Matthew Butkovic and Art Manion
In this webcast, Grace Lewis and Ipek Ozkaya provide perspectives involved in the development and operation of ML systems.
What attendees will learn:
• Perspectives involved in the development and operation of ML systems
• Types of mismatch that occur in the development of ML systems
• Future work in software engineering for ML systems
Are the great programmers really 10 times faster than the rest? What does this difference in productivity even mean? What productivity distribution should we expect between professionals? How can we use this knowledge? In this webcast, we make the most of a large set of programmer training data using repeated measures to explore these questions.
What attendees will learn:
• For routine tasks, professional programmers have a narrower range of productivity than we first supposed, but almost half of the variation in individual productivity is noise, making programmer rankings suspect.
• Rather than finding the “fastest” programmers, we should find competent people and give them the training and environment they need to succeed.
In this webcast, Carol Woody and Rita Creel discuss how cybersecurity engineering knowledge, methods, and tools throughout the lifecycle of software-intensive systems will reduce their inherent cyber risk and increase their operational cyber resilience.
This webcast illustrated where machine learning applications can be attacked, the means for carrying out the attack and some mitigations that can be employed. The elements in building and deploying a machine learning application are reviewed, considering both data and processes. The impact of attacks on each element is considered in turn. Special attention is given to transfer learning, a popular way to construct quickly a machine learning application. Mitigations to these attacks are discussed with the engineering tradeoffs between security and accuracy. Finally, the methods by which an attacker could get access to the machine learning system were reviewed.
Speaker: Dr. Mark Sherman
One of the primary drivers of the Department of Defense (DoD) Cybersecurity Maturity Model Certification (CMMC) is the congressional mandate to reduce the risk of accidental disclosure of controlled unclassified information (CUI). However, a full CMMC assessment can seem daunting to organizations in the Defense Industrial Base (DIB), and many might not know where to start. In this webcast, Model Architects Gavin Jurecko and Matt Trevors reviewed several steps for identifying CUI exposure in terms of their critical services and the assets that support them. This approach can help DIB organizations properly scope a CMMC assessment and contain the costs of protecting CUI.
Risk managers must often sift through the cacophony of demands for resources and advocacy to identify a diverse set of risks to include in their organization’s risk register. These managers of cyber risk face this problem when trying to prioritize risks within the scope of their function, only to then turn to executives and justify the need for resources.
OCTAVE FORTE, a new and upcoming Enterprise Risk Management (ERM) process model developed by Carnegie Mellon’s CERT Division of the SEI, provides a scalable and standardized process that assists managers and with policy guidelines and tools necessary for identifying risks and justifying the resources needed for the organization’s proper response to them.
Attendees at the OCTAVE FORTE webcast learn more about the new OCTAVE FORTE process and learn about a report, Advancing Risk Management Capability Using the OCTAVE FORTE Process, due this Fall. More specifically, the webcast attendees can expect to learn about the fundamental steps of the process and how they might apply them in their own organization.
|
|
|
|
| Bringing computation and data storage closer to the edge, such as disaster and tactical environments, has challenging quality attribute requirements. These include improving response time, saving bandwidth, and implementing security in resource-constrained nodes. In this webcast we review characteristics of edge environments with a focus on architectural qualities. The characteristics and quality attribute concerns that we present are generalized from and informed by multiple customer experiences that we have undertaken in recent years. We present an overview of edge environments, in both military and civilian contexts, and provide a discussion about edge-specific challenges and how they can differ based on the context. We discuss architectural quality attributes that are well suited to address the edge-specific challenges, and provide examples of how each apply. A microservices architecture provides an opportunity to address several of the quality attribute concerns at the edge. Through a final consolidated scenario as an exemplar, we discuss how the presented qualities can be addressed using microservices. This webcast should be useful for anyone interested in better understanding the challenges of edge environments and learning about representative scenarios of work currently being done. |
|
|
|
|
This webcast provided practical insights into how a Government Program Office can productively engage with a contractor using Agile and Lean methods. By reorienting the Agile Manifesto for a system acquisition context, we will consider the distinction between oversight and insight then briefly share examples of the impact of continuous delivery on technical review, requirements, testing, and system engineering.
Disruptive events and crises have the potential to irreparably harm your organization. The key to thriving, not simply surviving, in uncertain times is analysis of posture and preplanning. An organization can demonstrate operational resilience, when faced with both cyber and physical disruptions, if it focuses on the fundamentals and makes data-driven risk decisions.
The chasm between what academia researches and what industry uses in cyber is wide. By building mutually beneficial collaborations between the two, we can improve algorithms, datasets and techniques that are applicable to the real-world. Students and researchers should build a solid partnership with professionals early in their career to be exposed to and ground their work in current industry challenges. This ultimately results in more research being transformed into practical solutions.
Collaborations between the academia and the industry is one of the best ways for the industry to direct academic research outcomes to solve current problems. Without collaborations it can be challenging for the academia to produce algorithms, datasets and techniques that are directly applicable for real-world problems. Students and researchers have to build a working loop with the professionals early in their carrier to maximize the relevance of their work in practice, which ultimately results in more research being transformed to practical solutions.
The concept of software architecture as a distinct discipline in software engineering started to emerge in 1990 — although the idea had been around for much longer. Throughout my career in industry, then in academia, I’ve witnessed the growth of software architecture, its evolution in leaps and bounds. I’ve also had the privilege to meet and work with many of the key contributors who over 30 years have shaped it to what we know today: a mature discipline. It has its theories, its standards, its processes and tools, its place in schools’ curricula. Industry and academia, although often on different tracks —and often ignoring each other— have been making every year more incremental progress and even branching out subdisciplines or different schools of thoughts. But the obvious question is: are we done? what’s next? Plateau, obsolescence, retirement? Not quite. New problems arose, driven by new technologies, and some old problems were not really fully solved, or their context significantly evolved. In this brief talk, I’ll reflect on these 30 years, and pulling out my crystal ball, I’ll speculate potential developments ahead, from 4+1 different viewpoints.
SEI Chief Technology Officer Tom Longstaff interviewed Jeff Boleng, a senior advisor to the U.S. Department of Defense, on recent DoD software advances and accomplishments. They discussed how the DoD is implementing recommendations from the Defense Science Board and the Defense Innovation Board on continuous development of best practices for software, source selection for evaluating software factories, risk reduction and metrics for new programs, developing workforce competency, and other advancements. Boleng and Longstaff also discussed how the SEI, the DoD’s research and development center for software engineering, will adapt and build on this work to accomplish major changes at the DoD.
In an increasingly cloud-native world, application containers and microservice architectures are the next go-to for system architecture modernization. Like many technology choices, there are trade-offs that have to be carefully considered. Will containers solve my business problems? How will certain responsibilities shift between my software teams? How do I maximize my cyber security posture? Will I need to re-train staff? What is my budget for infrastructure and prototyping? In this webcast, David Shepard and Aaron Volkmann discussed some of the potential pitfalls of using containers and provide some food for thought to software teams considering embarking on a journey to containers.
You may have a secure application today, but you cannot guarantee that it will still be secure tomorrow. Application security is a living process that must be constantly addressed throughout the application lifecycle. This requires continuous security assessments at every phase of the software development lifecycle (SDLC). The SEI has researched a continuous authorization concept—DevSecOps—that allows for constant interaction between developers and information security teams throughout the entire SDLC. This allows any authorizing officials, such as personnel on information security teams, to be in constant contact with developers as changes are made to existing code and as new features are added. From project conception, a developed system security plan should be integrated into the development platform as well as other environments, where both developers and IAs can see the same artifacts for every development and deployment activity. This allows any changes to the system's security posture to be immediately identified and reported to the IA to evaluate and ensure that all security controls are adequately addressed. As a result, all security features can be verified and authorized, and eventually the organization will build a trusted culture among all stakeholders.
Hasan Yasar and Eric Bram discussed how the continuous aspect of communication and collaboration among developers and information security teams reinforces core DevOps principles, as well as allowing developers to write code with a "secure” development mindset. Giving developers and DevOps engineers the tools and knowledge to excel in their roles not only leads to enhanced productivity but also a more robust and secure application and environment development mindset. Giving developers and DevOps engineers alike the tools and knowledge to excel in their roles not only leads to enhanced productivity but also a more robust and secure application and environment.
In this webcast, CMMC Architects, Gavin Jurecko & Matt Trevors provide insight on how to evaluate and assess your organization’s readiness for meeting the practice requirements of CMMC Level 1.
Learn more about the DIB CS Program at: https://dibnet.dod.mil/
Or email: osd.ncr.dod-cio.mbx.dib-cs-ia-program-registration@mail.mil
CISA CRR Resources: https://www.us-cert.gov/resources
CMMC Accreditation Body – https://www.cmmcab.org
NIST SP 800-171A - https://csrc.nist.gov/publications/detail/sp/800-171a/final
Andrew Hoover and Katie Stewart discussed the DoD’s new CMMC program. They gave a brief overview of CMMC followed by a deep dive into the Process Maturity aspect of the model. The webcast provided insight into how organizations can prepare for CMMC.
This webcast will assist professionals and executives communicate risk concerns despite the cacophony and distraction posed by technical details and other organizational demands using the new OCTAVE FORTE approach. Practical tips for risk appetite development and application will be discussed.
This webcast covered the implementation of an automated, continuous risk pipeline that demonstrates how cyber-resiliency and compliance risk can be traced to and from DevSecOps teams working in the SDLC program and project levels. It will include integration of asset management, DevSecOps tooling, policy-to-procedure platform and risk management platform.
For more than two decades, Carnegie Mellon University’s Software Engineering Institute (SEI) has been instrumental in the creation and development of the field of software architecture. In our past webcasts, What Makes a Good Software Architect? (https://www.youtube.com/watch?v=CbLJC...) and What Makes a Good Software Architect (2019 Edition)? (https://www.youtube.com/watch?v=UFqys...), we have discussed what makes a good software architect. The range of knowledge and skills involved can be daunting, particularly given the pace of change in technologies and practices. In this session, a panel of architects will discuss their personal paths to becoming software architects and how they have helped others on that journey.
Artificial intelligence (AI) holds great promise to empower us with knowledge and scaled effectiveness. To harness the power of AI systems, we can—and must—ensure that we keep humans safe and in control. This session will introduce a new user experience (UX) framework to guide the creation of AI systems that are accountable, de-risked, respectful, secure, honest and usable.
In this webcast, as a part of National Cybersecurity Awareness Month, our experts will provide an overview of the concept of cyber hygiene, which bears an analogy to the concept of hygiene in the medical profession. Like the practice of washing hands to prevent infections, cyber hygiene addresses simple sets of actions that users can take to help reduce cybersecurity risks. Matt Butkovic, Randy Trzeciak, and Matt Trevors will discuss what some of those practices are, such as implementing password security protocols and determining which other practices an organization should implement. Finally, they discuss the special case of phishing—which is a form of attack that can bypass technical safeguards and exploit people’s weaknesses—and how changes in behavior, understanding, and technology might address this issue.
What attendees will learn
• Key findings from the CERT Division of the SEI, and the CERT-RMM team, in identifying commonalities among cyber practices and aligning them to CERT-RMM practices
• The CERT Division’s 11 cyber hygiene areas, comprising 41 CERT-RMM practices that are paramount to every organization’s success
• What organizations can do to change behavior, understanding, and technology to implement good cyber hygiene
Misuse of authorized access to an organization’s critical assets is a significant concern for organizations of all sizes, missions, and industries. We at the CERT National Insider Threat Center have been collecting and analyzing data on incidents involving malicious and unintentional insider since 2001, and have worked with numerous organizations across government, industry, and academia to develop and validate controls and best practices to address these concerns. In this webcast, as a part of National Insider Threat Awareness Month, our experts provided an overview of the ongoing research in this area, and answered questions about how the threat landscape continues to evolve, and what organizations can and should do to address insider threats.
What attendees will learn:
• Key findings from the CERT National Insider Threat Center’s research into the different types of insider incidents – motivations, vulnerabilities, and common attack paths
• How the insider threat landscape has changed over time, and what’s to come in the future
• What organizations can do to deter, detect, and mitigate insider threats from employees and trusted business partners
Ritwik Gupta and Elli Kanal explain what ransomware is, what it can do to your computer, and how you can help prevent infections using the concept of cyber hygiene. Ransomware is a type of malware that encrypts the files on a computer, preventing the user from accessing them. The attacker then extorts the user by requesting a ransom in exchange for the key that unlocks the files.
In this Cyber Talk episode, Ritwik Gupta and Elli Kanal explain how ransomware can infect a computer, and they discuss examples of how criminals have targeted single computers as well as large systems to explain what can happen when ransomware infects a system. To prevent ransomware attacks, Gupta and Kanal explain the concept of “cyber hygiene,” which refers to a set of basic practices that users can perform to decrease the risk of getting infected by malware. They stress the importance of developing an awareness for cyber hygiene, especially after the advent of the Internet of things, which has increased the number of devices that are susceptible to infection, including phones, cars, refrigerators, and more.
Rotem Guttman and Zach Kurtz explain what deepfakes are, how they work, and what kind of content it’s possible to create with current techniques and technology.
The term “deepfake” refers to the use of machine learning to produce content for essays or to modify photos and videos. When it comes to photos and videos, the images are often so realistic that viewers are not able to tell that they are fake. In this Cyber Talk episode, Rotem Guttman and Zach Kurtz explain the kinds of machine learning that people use to create deepfakes, how they work, and what kind of content it’s possible to produce with current technology. Rotem and Zach also cover the techniques people use to create fraudulent content. Such techniques include using an actor to film a video and then replacing the actor’s face with someone else’s, as well as more advanced methods that can reproduce a person’s body movements, voice, speech, and facial expressions to make that person appear to say or do something that he or she did not actually say or do. Finally, they discuss the current limitations of these technologies and techniques, and they forecast advances that might occur in the coming years.
Rotem Guttman and April Galyardt describe how machine learning (ML) fits into the bigger picture of artificial intelligence (AI) and discuss the current state of AI.
Currently, there is an enormous amount of interest in machine learning and artificial intelligence and what these new technologies can create for the present and future. In this SEI Cyber Talk episode, Rotem Guttman and April Galyardt discuss how machine learning fits into the bigger picture of artificial intelligence. They describe some of the current applications for machine learning as well as some of its limitations, including examples of machines reaching unexpected results, producing miscalculations because of contextual changes in the data they analyze, and introducing bias into their calculations. The participants also discuss possible use cases for and changes to machine learning that could occur in the mid to near future, including how machine learning might describe and explain its analyses for users to take appropriate action or to learn why the machine made certain decisions.
Recently, the Department of Homeland Security (DHS) released a warning about DNS hijacking and how website owners can protect themselves against it. To explain what DNS hijacking is and how adversaries use it to steal sensitive information, Elli Kanal and Daniel Ruef give a high-level overview of how DNS and network traffic work. They discuss how servers communicate with each other, what kind of information servers send to each other and why, and how adversaries can hijack that information. Finally, Elli and Daniel give some advice about what website owners might do to monitor their websites to make sure that adversaries have not hijacked their DNS.
In 2011, the Office of Management and Budget (OMB) issued the “Cloud First” policy to reform federal information technology management, which required agencies to evaluate cloud computing options. In 2012, the DoD Cloud Computing Strategy evolved to identify the most effective ways for the department to capitalize on opportunities and take advantage of cloud computing benefits that accelerate IT delivery, efficiency, and innovation as an enterprise. In the years since, many cloud transition efforts in both federal agencies and the DoD have experienced significant issues. This webinar will address a few of the causes for the transition issues, as well as identify some practices that will assist organizations as they plan to transition assets and capabilities to the cloud. The webinar will wrap up with a brief discussion of the 2019 Federal Cloud Computing Strategy – Cloud Smart, an updated cloud policy to improve cloud adoption for federal agencies developed by OMB.
As every software engineer knows, writing secure software is an incredibly difficult task. There are many techniques available to assist developers in finding bugs hiding in their code, but none are perfect, and an adversary only needs one to cause problems. In this talk, we’ll discuss how a branch of artificial intelligence called Natural Language Processing, or NLP, is being applied to computer code. Using NLP, we can find bugs that aren’t visible to existing techniques, and we can start to understand better what our computers are creating. While this field is still young, advances are coming rapidly, and we talk about the current state of the art and what we expect to see in the near future.
Today's DoD software development and deployment is not responsive to warfighter needs. As a result, the DoD's ability to keep pace with potential adversaries is falling behind. In this webcast, panelists discuss potential enablers of and barriers to using modern software development techniques and processes in the DoD or similar segregated environments. These software development techniques and processes are as commonly known as DevSecOps.
In 2017, the Software Engineering Institute (SEI) Webcast, What Makes a Good Software Architect? (https://www.youtube.com/watch?v=CbLJC...) explored the skills and knowledge needed by successful software architects. The architect’s role continues to evolve; in this webcast we revisited the question in the context of today’s role and responsibilities. We explored the challenges of working in an environment with rapidly evolving technology options, such as the serverless architecture style, and the role of the architect in Agile organizations using DevSecOps and Agile architecture practices to shorten iterations and deliver software faster.
Cybersecurity operators have to keep up with a world that's constantly changing, and they may lack the tools, time, and access to learn how to face actual threats. Simulated environments may not appear or behave the way they do in real life, and classroom-based approaches don’t provide the big picture. Throughout this talk, our team of researchers and engineers discuss the solutions we developed to help achieve a new level of realism in simulated cyber environments. Specific solutions include better Internet emulation, improved live network traffic, and human-like behavior of host systems. This set of tools recreates the real world in a controlled environment, providing the platform where cyber operators can enhance their security skills.
Attendees will learn how to
• enhance a bare-bones, cyber-emulation environment using open source tools
• provide the best training possible by simulating your own networks so that employees can learn how to respond to real-world threats
• get help from the SEI to implement these tools in your own environment
In this webcast, Lori Flynn, a CERT senior software security researcher, describes the new features in SCALe v3, a research prototype tool. SCALe v2, available on GitHub, offers a subset of features available in SCALe v3. Over the last three years, as part of alert classification and prioritization research projects she has led, her team has added new features to the (privately released) 2015 version of SCALe (v1) that are intended to assist with automated static analysis alert classification and advanced alert prioritization. Flynn invites people in other organizations to collaborate with her team, including testing SCALe v3 and providing sanitized audit archives. Collaborators also might have an opportunity to become involved in developing a version of SCALe that would be usable in production, not just as a research prototype tool.
In this first webcast in a two-part series, April Galyardt and Carson Sestili described what metadata is and what information can be gleaned from it.
Social networks have become part of our daily lives. We browse, share, “like,” and generally communicate with friends using these tools every day. In the midst of all this, we rarely stop to consider how much information about ourselves we are freely handing over to the social network companies. This information, called “metadata,” contains an incredibly rich—and often frighteningly detailed—view of some of the most personal aspects of our lives.
In this first webcast in a two-part series, we described what metadata is and what information can be gleaned from it.
Specifically, we discussed:
• How metadata gets generated
• How it can be used to uncover extensive personal information
• Steps you can take to protect your privacy
In this webcast, we explain how the technology works and what makes it fundamentally different than its predecessors. We discuss where it fits (and where it doesn’t fit) and help set a rubric to help you determine if you need this technology.
In this webinar, a watch panel consisting of Robert Floodeen, William R. Claycomb, Andrew P. Moore, Kurt C. Wallnau, Randall F. Trzeciak, Alex Nicoll discuss Managing the Insider Threat: What Every Organization Should Know.
In this webinar, Randy Trzeciak discusses a study to develop insights and risk indicators related to malicious insider activity in the banking and finance sector.
In this webinar, Randy Trzeciak, Technical Manager of the CERT Insider Threat Center, described the summary of new requirements mandated by NISPOM Change 2 and the impact it will have on DoD contracting organizations.
In this webinar, Kurt Wallnau discusses insider threat controls and how to test systems whose dynamics are based in human nature that is only partially understood.
Watch Nader Mehravari discuss "Recent Federal Policies Affecting the Cybersecurity and Resiliency Landscape" from the SEI Virtual Event, CERT® Operational Resilience: Manage, Protect and Sustain.
In this webinar, Rich Pethia discusses how cybersecurity has changed over the past 20 years.
Watch Julia Allen discuss the United States Postal Inspection Service (USPIS) (Case Study) from the SEI Virtual Event, CERT® Operational Resilience: Manage, Protect and Sustain
Watch Matthew Butkovic discuss the "Department of Homeland Security Cyber Resilience Review (Case Study)" from the SEI Virtual Event, CERT® Operational Resilience: Manage, Protect and Sustain.
Watch James Cebula discuss the "Overview of the CERT® Resilience Management Model" from the SEI Virtual Event, CERT® Operational Resilience: Manage, Protect and Sustain
Watch Nader Mehravari discuss the "ABCs of Operational Resilience" from the SEI Virtual Event, CERT® Operational Resilience: Manage, Protect and Sustain.
This webinar covered the perspectives of security practitioners on building secure software using the DevOps development process and modern security approach.
In this webcast, we’ll discuss some of the factors holding blockchain back, as well as where they can expect to see it in the future.
In this webinar, Doug Reynolds and Aaron Volkmann discuss key DevOps principles, including cross-team collaboration, task automation, tool integration, continuous integration and deployment, and continuous monitoring. Doug and Aaron discuss how you can use these principles to maximize cybersecurity assurance, quality, and productivity. They also share real-world lessons-learned from their experiences in the field.
Watch Jeff Boleng, Robert Schiela, Samuel Procter, Lena Pons, and Nathan VanHoudnos discuss "Three Software Innovations that DoD Needs Now".
Watch this lively discussion in which we answered attendee questions on all things Agile and DevOps.
During this webinar, Dr. Shannon examined the questions, science, and technology that builds trust with customers, other organizations, and society to ensure their security and privacy, and our own resilience and accountability.
Have software's repeated successes, and the assumption that they will continue endlessly, discounted perceptions of its importance among leadership in civilian government, national defense, and national security organizations?
This webinar focused on the development and application of combined data analytics and offered several examples of analytics that combine domain resolution data, network device inventory and configuration data, and intrusion detection.
Watch this webcast to gain insights into effective metrics programs in government settings.
In this discussion-focused webinar, Bob Binder and SuZ Miller will discuss 5 key questions that government organizations contemplating embarking on adopting automated test techniques and tools in an Agile environment are likely to have.
In this webinar we discussed the challenges facing the CRO role and about how CMU's new CRO program can help you address those challenges.
This webinar summarizes much of what the SEI has learned in its eight years of researching and facilitating adoption of Agile and Lean methods in software-reliant systems in government.
Network flow records provide a useful overview of traffic on a network that uses the Internet protocol (IP) to pass information. Huge numbers of bytes and thousands of packets can be summarized by a relatively small number of records, with few privacy concerns and a small record size (which aids both speed of retrieval and duration of storage). However, examining these records to build an awareness of the security situation on a network requires automation, and it can be daunting to develop a process for building the automated analytics. This webinar presents such a development process, outlining how to determine what to analyze, how to analyze it in an automated manner, and issues involved in validating and interpreting the results.
In this webinar, we discuss topics to consider when planning a large modernization project and share mitigation strategies for executing the modernization effort.
This webinar describes some of the issues involved in automating the collection and analysis of malware, which has seen exponential growth over the past decade.
For different reasons, usability is generally an afterthought in the cybersecurity tool development process. In this webinar, we teach the audience the value of defining the problem and how this impacts the software quality outcomes.
In this webinar, we discussed how you can improve your organization's secure coding capabilities and how to improve your workforce, processes, and tools to develop and verify the security of your software before it is deployed.
Over the past few years, there has been a veritable explosion of hiring in the field of data science. Just ten years ago, the phrase data scientist was almost unheard of; nowadays, data scientist positions are advertised across numerous industries, with a particular focus on high tech. What is this position and why is it relevant? In this webinar, we discussed this position from a number of angles—what the term “data science” means, what skills a data scientist brings to the table, what competitive edge data science can bring to your team, and the differences between data science and business analysis. We also discussed a number of case studies that describe how data science can be integrated into existing businesses as well as how to best make use of data scientists’ skills.
Learn why secure coding practices are important to reduce common programming errors that lead to vulnerabilities.
Learn the importance of developing security requirements in the same time frame as functional requirements.
Learn how to better identify process improvements at your organization through new perspectives on secure software development and delivery.
Learn how to develop a vulnerability coordination capability, which helps you respond to vulnerabilities and demonstrates that you are serious about fixing them.
Last summer's Wired article describing vulnerabilities in the Jeep shows that software is being created and deployed with exploitable, yet avoidable, security flaws. So far, the automotive attacks have been largely demonstrations. However, successful cyber-attacks have been carried out on steel furnaces in Germany and the electrical grid in Ukraine. Insecurely written software in cyber-physical systems places people and property in jeopardy. Fortunately, there are many techniques available to those building software for cyber physical systems that can greatly reduce their vulnerability. This talk starts with an in-depth review of the Jeep scenario. It then examines how security can be introduced throughout the software development lifecycle to blunt such vulnerabilities.
For two decades, the SEI has been instrumental in the creation and development of the field of software engineering known as software architecture. An architect whose skills and capabilities match a project's needs is more likely to be successful. So what are those skills? In this webinar, SEI researchers and an industry colleague discussed in two talks What Makes a Good Software Architect? John Klein and Andrew Kotov on Skills and Knowledge of Successful Architects Ipek Ozkaya and Michael Keeling on Architects Design Trade-off Toolbox: Balancing Agility and Technical Debt What viewers will learn: How the technical skills needed by a software architect change throughout a system's lifecycle and how this influences the architect's success How architects should be the champions of product quality while making the right (and timely) design trade-offs
This webinar proposed the Intelligence Preparation for Operational Resilience (IPOR) framework to create a model for structured analysis of your intelligence needs and a way to operationalize threat intelligence once you have received it. To build a structure to meet this end, the IPOR references and builds upon frameworks such as the military’s Intelligence Preparation of the Battlefield process and the CERT® Resilience Management Model.
Chief Information Security Officers (CISOs) are increasingly finding that the tried-and-true, traditional information security strategies and functions are no longer adequate when dealing with today’s increasingly expanding and dynamic cyber risk environment. Many opinions and publications express a wide range of functions that a CISO organization should be responsible for governing, managing, and performing. How does a CISO make sense of these functions and select the ones that are most applicable for their business mission, vision, and objectives?
This webinar describes a CISO organizational structure and functions for a typical large, diverse organization using input from CISOs, policies, frameworks, maturity models, standards, codes of practice, and lessons learned from major cybersecurity incidents.
The most precious resource people have in today’s ever-connected world is their attention. Human interaction with information systems has continually shrunk from hours behind a terminal, to minutes interacting with a cell phone screen, to only 3–5 seconds glancing at a smart watch or wearable device. The key to making information immediately understood and actionable in the era of glance-able interaction is understanding context and, eventually, user intent. People inherently understand context. They naturally apply the context of a situation or conversation during everyday activities. Information systems, on the other hand, apply a user’s context to the task at hand in very different ways. To improve the way that information systems use context, our research has focused on individual and group context supporting dismounted soldiers and first responders operating in edge environments. The goal of our research in context-enabled computing is to ensure that the right information is delivered to the right individual at the right time to ensure mission success. In this webinar, we will cover a wide variety of research activities associated with our efforts to better leverage context for information delivery and sensor tasking.
Cyber situational awareness is an emerging topic in network operations and defense, yet the overarching concept of situational awareness has been widely used and studied extensively for decades. During this webinar, we will • describe the foundations of cyber situational awareness • explore how to apply situational awareness concepts to the cyber domain • look at how network flow plays a critical part in gaining situational awareness over today’s complex networks • introduce tools that can be used to collect and analyze network flow data • review examples that show the successful use of network flow to solve operational and security problems
The implementation of DevOps implies improvement across the entire scope of software delivery. However, as with any process change or introduction of new technology, lack of attention to security can invite disaster. In this presentation, we’ll explore some of the security-related topics and expectations that can be addressed when planning and changing your process to accommodate DevOps practices. Specific topics will include automation, monitoring, team collaboration, customer interaction, and also specific technologies such as Docker and Guantlt.
A surprisingly large number of different types of testing exist and are used during the development and operation of software-reliant systems. We have identified nearly 200 of these general types of testing and there are many additional types that are application-domain specific. While most testers, test managers, and other testing stakeholders are quite knowledgeable about a relatively small number of testing types, many people know very little about most of them and are unaware that others even exist. One way to understand so many types of testing is to classify them into a taxonomy that groups similar testing types together. One way to organize them is by the types of questions they answer. Specifically, types of testing can be categorized by the five Ws and two Hs: what, when, why, who, where, how, and how well. Understanding these different types of testing is important because different types of testing tend to uncover different types of defects and multiple testing types are needed to achieve sufficiently low levels of residual defects. Whereas not all of these testing types are relevant on all projects, a complete taxonomy can be very used to help discover the ones that are appropriate and ensure than no relevant type of testing is accidentally overlooked. Such a taxonomy can also be useful as a way to organize and prioritize one's study of testing. This tutorial introduces the attendee to our taxonomy of testing types, thereby clarifying the grand scope of testing and enabling the attendee to better select the appropriate types of testing to for their specific needs.
SEI Chief Operating Officer, Robert F. Behler discusses Cyber-Vulnerabilities in Aviation Today.
Will Dormann discusses a tool that shows whether a connection to the web is secure and what information is being transmitted.
Rhiannon Weaver discusses how a small subset of features from dynamic malware analysis can help to uncover possible relationships among files and to direct static reverse engineering efforts.
Jose Morales discusses mobile device security enhancements with defensive and offensive uses.
Greg Shannon discusses the CERT Division's current work associated with cyber community of interest (COI).
Will Klieber and Lori Flynn discuss undesired flows of sensitive information within and between Android apps.
CERT researchers discuss DevOps and its relationship to cybersecurity and the dynamic threat.
About the Webinar DevOps is all about delivering business value as rapidly as possible. Embracing its philosophies goes beyond implementing automation and tooling to speed software development and delivery. DevOps is a culture of communication and collaboration. For many of us, shifting to this new culture can create organizational "culture shock," or discomfort by those suddenly subjected to an unfamiliar culture, way of life, or set of attitudes. DevOps is not something you purchase or have implemented by a DevOps Engineer. Instead, a shared vision and common goals across teams and team members are critical to making a successful culture transformation. Join us for a discussion about ways to shift organizational culture to achieve DevOps. We will highlight communication tools and movements, such as ChatOps. DevOps is all about teams working together to continually improve their processes and achieve their shared business goals. With cross-functional teams, documentation must be in place. We will discuss strategies for curating information repositories that fit how a team works. Also, project and team management are core components of ensuring that team members are able to work efficiently and don’t get lost in meetings and context switching. Automated tools further enhance communication by exposing project data to everyone.
The definition of DevOps is a highly contested topic. Despite what some will lead you to believe DevOps is not just a set of tools, nor is it merely a focus on achieving continuous integration, continuous delivery, or continuous deployment. DevOps practices enable a team to achieve the level of coordination and understanding necessary to realize all of these goals, as well as to automate infrastructure, testing, and deployment. Specifically, DevOps provides organizations a way to achieve • collaboration between project teams and roles • infrastructure as code • automation of tasks, processes, and workflows • continuous monitoring of applications and infrastructure Business value drives DevOps development. Without a DevOps mindset, organizations often find their operations, development, and testing teams working toward short-sighted incentives of creating their infrastructure, test suites, or product increment. In this webinar, we’ll talk about DevOps, its common misconceptions and roadblocks, and how you can use DevOps to help your organization reach new heights of efficiency and productivity.
While software security is an increasing concern for software and system architects, few architects approach this quality concern strategically. Architects and developers primarily focus on functionality, and security is often applied as a band-aid solution after an application has been developed. In the second talk we report on three case studies of real-world projects—two industrial and one open-source—where we attempted to measure the consequences of various architectural approaches to security. The results of our case studies indicate that a strategic, system-wide, architectural approach to security, implemented as a security framework, results in the best outcome from both security and maintenance cost perspectives.
Software architecture has enormous influence on the behavior of a system. For many categories of systems, early architectural decisions can be a greater influence on success than nearly any other factor. After more than twenty years of research and practice, the foundations for software architecture have been established and codified, but challenges remain. Among other trends, increased connectivity, a shift to the cloud and to mobile platforms, and increased operational and market tempos have precipitated the need for changes in architectural practices and decisions. The first talk shares a perspective on the trends influencing the need for change, the related architectural challenges, and the applicable research and practices.
Sound cyber intelligence practices can help organizations prevent or mitigate major security breaches. For several years, researchers at the SEI have been examining methodologies, processes, technology, and training to help organizations understand what it means to perform the work of cyber intelligence. To spur further development and advance understanding in this important area, the SEI launched the Cyber Intelligence Research Consortium, aimed at helping organizations make better judgments and quicker decisions related to cyber intelligence. The consortium's membership consists of practitioners and decision makers from multiple sectors, including government, energy, banking, defense contracting, and academia. At this webinar SEI practitioners will provide a detailed overview of the consortium's work during this inaugural year. Efforts for this year include how-to guides for navigating key analytical practices and technologies, tradecraft labs, and an in-person crisis simulation event. Webinar participants will also receive a guide for evaluating intelligence they can use in their own organizations.
Soldiers and front-line personnel operating in tactical environments increasingly make use of handheld devices to help with tasks such as face recognition, language translation, decision making, and mission planning. These resource-constrained edge environments are characterized by dynamic context, limited computing resources, high levels of stress, and intermittent network connectivity. Cyber-foraging leverages external resource-rich surrogates to augment the capabilities of resource-limited devices. In cloudlet-based cyber-foraging, resource-intensive computation and data are offloaded to cloudlets. Forward-deployed, discoverable, virtual-machine-based tactical cloudlets can be hosted on vehicles or other platforms to provide infrastructure to offload computation, provide forward data staging for a mission, perform data filtering to remove unnecessary data from streams intended for dismounted users, and serve as collection points for data heading for enterprise repositories. This webinar presents the tactical cloudlet concept and experimentation results for five different cloudlet provisioning mechanisms. The goal is to demonstrate that cyber-foraging in tactical environments is possible by moving cloud computing concepts and technologies closer to the edge so that tactical cloudlets, even if disconnected from the enterprise, can provide capabilities that enable enhanced situational awareness and decision making at the edge.
In this webinar, John Haller and Matthew Butkovic of the CERT Division of the Software Engineering Institute will discuss real-world incidents, including recent industrial control system attacks and incidents affecting Department of Defense capabilities, and the lessons that organizations should take away. The session will focus on the lifecycle of supply chain relationships and introduce concepts to help organizations manage them more effectively. Managing the risks of depending on external entities and supply chains to support critical services has increasingly become an area of concern for both the federal government and private critical infrastructure organizations. External dependencies may consist of business partners that your organization relies on, cloud services such as data processing, or storage facilities. Or these dependencies may take the form of reliance on public infrastructure such as transportation or the electrical grid. The webinar speakers, John and Matthew, will discuss the HAVEX malware attacks on industrial control system vendors, which were reported to the security community in June 2014. For supply chain risk management, a key lesson from the HAVEX case is the importance of having a process to identify and prioritize external dependencies. The speakers will also explore and discuss methods for addressing this problem in a realistic, reliable way. Also covered in the webinar are the lessons for third-party risk management that organizations should take away from recent attacks on DoD-affiliated transportation contractors. The speakers will explain how to correctly scope and build security programs around key, organizationally critical services. The speakers will discuss how your organization can learn from these incidents, including best practices around forming relationships with external entities and managing the relationship over time to support your organization's incident management and situational awareness processes. The webinar closes with a recap of key supply chain risk management capabilities and an update to CERT research into the state of these capabilities across U.S. critical infrastructure sectors.
Most software systems have “defects” identified by users or developers. For most systems, it is too costly to fix all of the concerns in the near term, and indeed some issues may never be addressed. The government program office (or other procuring organization) has an obligation to choose wisely among a set of competing defects to be repaired, especially in a financially constrained environment. RPN can help by quantifying three distinct attributes of failure in a composite measure which helps to structure objective analysis and decision making. This webinar will explain the component used in RPN and how it can help a program select between competing defects to best utilize constrained resources to help lower overall system risk.
Safety-critical systems, such as those used in avionics and the medical and aerospace domains, are becoming increasingly reliant on software. Malfunctions in these systems can have significant consequences, including mission failure and loss of life. As a result, they must be designed, verified, and validated carefully to ensure that they comply with system specifications and requirements. A car contains many electronic control units (ECUs)—today’s standard vehicles can contain up to 30 ECUs—that communicate to control systems such as airbag deployment, antilock brakes, and power steering. The design of tightly coupled software components distributed across so many nodes may introduce problems, such as early or late data delivery, loss of operation, or concurrent control of the same resource. In addition, errors introduced during the software design phase, such as mismatched timing requirements and values beyond boundaries, are propagated in the implementation and may not be caught by testing efforts. If these problems escape detection during testing, they can lead to serious errors and injuries. Also, because such systems are designed to be operational for many years, errors are often found when reviewing code from legacy systems designed and built more than 20 years ago and still operating, as in the avionics and aerospace domains. Unfortunately, late discovery of errors leads to major rework efforts and often postpones product delivery. Such issues are not specific to a particular domain and may occur in all in safety-critical systems. During the last 10 years, SEI researchers have been working on methods, languages, and tools to design safety-critical systems, find potential issues at the earliest phase in the development process, and avoid potential re-engineering efforts. Our techniques help system architects design the system and check requirements enforcement without having to implement the system. This webinar introduces the Architecture Analysis and Design Language (AADL), the architecture modeling language used to specify safety-critical systems. We show its use in the Open Source AADL Tool Environment (OSATE) to design and validate a generic automotive application—a speed-regulation system. In particular, this webinar will demonstrate analysis capabilities of the tool for different perspectives, including resources budgets, performance/latency, and safety.
Watch Dave Zubrow discuss "Taking Advantage of Agile while Minimizing Risk" at the Agile for Government Summit. The purpose of this event was to: foster better understanding of how agile software development methods are providing the basis for incremental and modular acquisition across Government to discuss the changing technology of modern information-intensive businesses and the implications these have for evolving Government approaches to streamlining IT. This Summit is a partnership between the AFEI ADAPT and the Carnegie Mellon University Software Engineering Institute.
On April 25, 2014, technical staff from the Software Engineering Institute (SEI) and Codenomicon participated in a live-streamed panel discussion on the impact of the Heartbleed OpenSSL vulnerability along with methods to mitigate and even prevent crises like this in the future. Chris Clark, Security Engineer from Codenomicon, one of the cybersecurity organizations that discovered the Heartbleed vulnerability, joined members of SEI's technical staff from the CERT and Software Solutions divisions and from the SEI's Information Technology department. They will be discussing how software vulnerabilities like Heartbleed can be mitigated through the different phases of the secure software lifecycle using techniques available today. They will also discuss how changes to our current software development and management techniques need to be managed to more effectively reduce the effects of incidents like Heartbleed.
Watch Ipek Ozkaya discuss “Why Should Government Care about Technical Debt and Software Architecture?” at the Agile for Government Summit. The purpose of this event was to: • foster better understanding of how agile software development methods are providing the basis for incremental and modular acquisition across Government, and • to discuss the changing technology of modern information-intensive businesses and the implications these have for evolving Government approaches to streamlining IT. This Summit is a partnership between the AFEI ADAPT and the Carnegie Mellon University Software Engineering Institute.
What constitutes stellar performance and best practice? You can't really say what's good or best ... unless you measure it. High-performing athletes rely on measurement to understand and improve so that they can compete effectively and win. Can knowledge workers such as software engineers use measurement in a similar approach? Absolutely. But the measures need to be practical, relevant, trustworthy, and actionable. They need to be used by the individual to benefit the individual. Watch to: see the emerging empirical results of over 100 software project teams that have collected accurate performance data learn about the techniques that were developed and used to validate the accuracy of the collected data *learn how four basic measures can provide close-looped feedback to help software engineers understand and improve their performance You don't need to measure everything. It only takes a few basic and easy-to-collect measures to help you and your team manage schedule commitments and software quality. Find out what those key measures are, how you can collect them, and how you and your software development team can use them effectively. During this webinar, we shared the performance results of over 100 software teams that have carefully tracked their schedule performance and the quality of their work. We showed how high-integrity empirical results such as these can be used at the individual, project, and industry levels to characterize performance in meaningful and insightful ways.
Watch Randy Trzeciak and David Mundie discuss an "Overview of the Threat Posed by Insiders to Critical Assets" from the virtual event Managing the Insider Threat: What Every Organization Should Know. About the Speaker(s) Randy Trzeciak is Technical Manager of CERT’s Enterprise Threat and Vulnerability Management Team and the CERT Insider Threat Center at Carnegie Mellon University's Software Engineering Institute. The team’s mission is to assist organizations in improving their security posture and incident response capability by researching technical threat areas, developing and conducting information security assessments, and providing information, solutions and training for preventing, detecting, and responding to illicit activity. David Mundie is a member of the CSIRT Development Team within the CERT® Program at the Software Engineering Institute (SEI), a unit of Carnegie Mellon University in Pittsburgh, PA. He has been at CERT since 2000 and has worked in a variety of areas including insider threat, malware analysis, and incident management capability metrics. From 2006 to 2009, he was a member of the Q-CERT project, which established a national information security team for the country of Qatar.
The wide variety and different characteristics of NoSQL databases creates a complex technology acquisition and design landscape for organizations looking to build scalable, high performance data management systems. In addition, scalable 'big data' systems are significant long-term investments that must scale to handle ever-increasing data volumes, and therefore represent high risk applications in which the software and data architectures are fundamental components of ensuring success. This talk describe how we are developing a software and data architecture knowledge base and technology evaluation approach specifically targeted at big data systems and NoSQL technology adoptions.
The Agile movement began as a reaction to frustration over slow delivery of software which often didn't sufficiently meet user needs. DevOps picks up what Agile started. Software development velocity has improved in many cases, yet we see deployment-related delays due to issues such as inability to integrate continuously (or even frequently) resulting in late discovery of costly integration issues, challenges completing automated testing within an increment/build cycle and uncertainty about whether a build is stable and secure enough for external release. To avoid problems such as these we suggest it is critical for teams to make design decisions that align with their deployment goals such as reduced deployment cycle time and continuous delivery.
Watch the CERT® RMM User Panel discuss their experiences implementing RMM from the SEI Virtual Event, CERT® Operational Resilience: Manage, Protect and Sustain. Panelists inlcluded: Michael Ray of the United States Postal Inspector Service (USPIS), Kevin Dillon of the Department of Homeland Security (DHS), Jason Christopher of the Department of Energy (DoE), Christopher Burgher of SunGard, & William David of Lockheed Martin
Watch George Silowash and Alex Nicoll discuss best practices and controls for mitigating insider threats from the virtual event Managing the Insider Threat: What Every Organization Should Know.
In this 2013 webinar, Christian Roylo discusses the role of technology in responding to large-scale cyber incidents.
In this August 2013 webinar, Nader Mehravari discusses how to protect and sustain the mission and business operations of an organization.
In this 2013 webinar, Roman Danyliw discusses how cyber security organizations react to new technologies or adversaries.