Voice of the DBA: Recent Episodes
Steve Jones
A series of episodes that look at databases and the world from a data professional's viewpoint. Written and recorded by Steve Jones, editor of SQLServerCentral and The Voice of the DBA.
Brent Ozar has a series of database animations posts, where he tries to explain what work is done by SQL Server during certain operations, such as Index Seeks and Page Splits. These show how the engine might need to read or update various pages as it tries to perform operations. Both experienced and novice SQL Server people might think that these are interesting, but not that useful.
I think they're great.
Read the rest of Imagine the Physical World
Apparently, Meta did what a lot of employees suspect their management will do: use AI to lay off people. In this case, there is a report (and lawsuit) that Meta used it's AI-integrated HR platform to make decisions about who to let go in a layoff. The rumor is that the AI used productivity metrics to choose who was the target of the layoff. As with a lot of AI failures, this appears to be another case of poor communication guiding the AI, or the AI not actually taking individual situations into account.
A number of the people terminated were on maternity/paternity leave, which is a protected activity. Others may have been on other medical leave, though for privacy reasons, the article doesn't have firm data to prove this. The lawsuit will likely bring more of this out, but this appears to be a case not just of AI making decisions, but of poor behavior from management. Plaintiffs were discouraged from taking leave off, which is something sh****y humans have said to people for decades. We need you; your baby or family isn't important, so don't use your leave. It's one aspect of working in the US that is way worse than overseas, where there are more employee protections.
Read the rest of The Quiet Part
Recently, I noticed my son was coming home later and going to work earlier. He typically works a 9/80 schedule as a software engineer, but at times he might work extra hours on a deadline. His company tries hard to keep employees working a set schedule without overtime. Unless there is a need, after all, this is software. However, when they are on a deadline, management warns people, and they try to manage overtime to reasonable levels to avoid burning out employees.
I asked him if he was extra busy or if AI was encouraging extra hours. He told me this was a short-term project with a few more hours, but mostly the team was decompressing a bit after work. I completely understood that as I've spent my share of time with co-workers at theend ofday, sometimes sharing a drink or meal nearby, sometimes just chatting in the office or the parking lot.
Read the rest of Are You Working More Hours?
T-SQL Tuesday #200 was in July, hosted by Brent Ozar, and it was a great topic: How do you recognize a bad query? In the age of AI, when lots of people will get queries written by others (people or AIs), how can you easily and quickly review code? Review is already a challenge in the software world, and I am sure it's going to be even more challenging as people let machines author more database code.
Lots of you might hope that an AI agent will write better code than your average developer, but I don't know if I'd count on that. There is a ton of poor query examples on the Internet and that's where AI models are trained. You need some sort of feedback loop, good testing, and strong guidance if you want better query code. I think it's as likely as not that AIs will produce poor queries just like humans.
Just faster.
Read the rest of Finding Bad Queries
When we look at the performance of software, we use Pnn notation to indicate the latency of an issue. A P95 problem is one that exceeds the time that 95% of the other queries take. In other words, this is the 5% slowest things happening, which can include database slowdowns that impact your application.
For many years in software development, we have tended to work on the P95 or P90 issues, the slowest items. This is primarily because those can make a big difference to the system's performance. If I fix the slowest things, then the system feels faster. Certainly, I know lots of DBAs and developers will apply this logic to database queries. They focus on the slowest queries and tune them to improve the system. If most things are quicker, especially the things most users notice, then the system feels faster.
Read the rest of Fixing P1 Queries
Lots of AI usage has been spent inside companies on building new software. Sometimes people are trying to rebuild existing software. I've seen more than a few articles say that Slack is dead (it's changing), or Monday is dead, or some other SaaS isn't going to survive because people can vibe code their own replacement. I know a few people trying to do this, with very mixed success.
I saw this post from Jason Friend of Basecamp, where he noted that most products wouldn't exist if everyone were an entrepreneur. This is because the great products exist precisely because an entrepreneur had a great idea and followed it through. They made mistakes, they learned, and they spent a lot of time getting the product right. The coding was likely a part of this, but the mistakes, the changes of direction, the decisions on what to add or take away, those are time-consuming things not shortcut by AI. The coding isn't the big delay in lots of products. A great response to the post notes this: there are many decisions that go into a product. Many people don't think about all those decisions.
Read the rest of Building Your Own Software
The first one is hard. The rest are boring.
I heard this statement from someone recently, and it sounds like something a technical person would do. It's been a goal of mine, or maybe a direction to aim for, though I sometimes think that goal is more aspirational than actual. It can be hard to do that for every task.
The first time I tackle something, it should be hard. It's new work. It's a new process/code/thought/action/etc. It's unfamiliar, and I spend more time on it than I want. After that, I'd hope I could repeat the thing again in much less time. That's the goal, and that's what we aim for in a lot of DevOps work. Make the things we think are hard, less hard. Make them boring by codifying things, using automation, and have the computer replicate the task.
Read the rest of Make It Routine
This past week I saw an article on eWeek that the newest OpenAI GPT-5.6 (Sol) model has deleted local files and live data. Files I'm less worried about, but data concerns me. There are lots of file backups, and certainly version control should be enabled for any developer tasks on which an AI works. However, data is harder, since it can change quickly, especially in live environments.
There are different reports, some of which seem like more human error issues than the model's, but we should account for human error when we use models. In one report, a model had access to a live production database and cleared tables for integration tests. Database testing is hard, as I've learned over the years. Many developers don't think about how testing works with live data and how it is different from mocks and stubs. It is different, and if you use a lot of testing frameworks on live databases, you run the risk of there being issues. You could have data loss, and almost certainly will have some level of downtime disruption.
Read the rest of Another Model, More Data Loss
I had a request from a customer recently who asked if we could give them a report of their database server instances and include CPU usage. This request was filtered through an account executive, so something was lost in translation, but I was confused and asked for clarification, as asking for CPU usage is kind of like asking how fast you were traveling in your car. There needs to be more context.
If someone asked you for a report of CPU usage for a database, what would you expect? How would you report this? I'm sure the person asking might make a difference. A fellow DBA, your DBA manager, or maybe an executive could all view this differently. I want to know how things are performing, if there is a trend, or maybe if we are getting value for the hardware we've provisioned, depending on my role.
Read the rest of What is CPU Usage?
What if I got hit by a bus?
I'm sure many of you have heard some variation of this, or used it, or maybe your boss told you to be ready in case you (or someone more important) gets hit by that vehicle. This is the "bus factor," and it's often used in reference to preparation for unforeseen events. Not literally someone getting hit by a bus, but perhaps a sudden accident that takes someone away from work (hopefully not fatal). Perhaps it's someone leaving for a new job. Maybe it's retirement, which is something my boss and I at Redgate chat about periodically.
Don't worry, I'm not planning on retiring anytime soon. I expect to work until at least 2033 and possibly longer. Likely longer, I'm a weird work-a-holic in some ways, and I love my job.
Read the rest of The Mythical Bus Accident
The AI boom is still growing like crazy. Many organizations are trying to learn how they can use AI to improve operations and become more efficient, at a reasonable cost. Plenty of companies are spending crazy amounts of AI tokens, sometimes blowing their yearly budgets in months and not necessarily receiving substantial value back. Some companies are trying to train AIs to understand their operations and perhaps reduce their other costs, primarily labor. Still others are tip-toeing in the waters of AI LLM use and conducting smaller experiments, with limited access to AI technology.
Meta has been a company on the forefront of trying to train AI based on the work employees already perform. There has been plenty of concern that their efforts are designed to lower headcount and replace humans with AI agents. That might or might not work, though I don't expect a lot of organizations to do this. It's likely harder than any of the hype suggests, and most organizations have much more complex types of operations than Meta.
Read the rest of Security and AI Fail
I recently recorded a session with Ken Muse and a Redgate Flyway Solution Engineer. It was a fun session using GitHub and AI, and better managing the code in an automated fashion to bring some determinism to AI coding. I'm hoping it will be released soon, and you can see a vision of how you can better wrangle your AI agents and reduce risk and increase reliability.
When we started our discussion, Ken noted that he is an AI forward deployed engineer for GitHub. His job is to work with teams in how to use agentic coding. When we were first prepping, I had never heard the Forward Deployed Engineer title, which is apparently getting popular. It was in an issue of the Pragmatic Engineer Deep Dives last year, and I must have missed that issue. Apparently, this is a role that works part of the time with customer teams and part of the time with product or engineering teams.
In other words, a software engineer with a new title who gets paid more than a software engineer.
Read the rest of Forward Deployed Engineers
Oh, how I wish I could make you learn. How I wish I could coach, guide, inspire, or even bribe you to learn more about your job, or things related to your job, or even things in life. I wish all of you would improve your skills, but more, I wish you would want to improve your skills. I find lots of people who do want to get better, but far too often, people aren't trying to improve because they want to coast along at their jobs.
I get it. You're stressed and busy at work, though hopefully not too often. You have challenges at home, kids to raise, parents getting older, financial stresses, concerns about politics or sports or exercise or diet or just about anything in the world. We all have things that take mental energy in our lives. How/why/when should I add another thing to the list?
Read the rest of I Can't Make You Learn
I saw this article and thought, surely there's nothing I could use in here: 10 Best AI Prompts for Everyday Tasks. Quite often, when these articles appear, they are very high-level and contrived examples of things that I rarely find myself doing.
So I clicked the link.
Read the rest of A Quick Second Opinion
Tomorrow is the United States 250th Independence Day celebration. I've been hearing about this on the radio for weeks, and it's struck me that I've been alive and lucky enough to experience both the 200th (1976) and the 250th (2026) milestones. I'm not sure either is more amazing than any other 4th of July, but the round numbers stand out.
Read the rest of Independence Day.
Satya Nadella talked about cognitive coverage in the age of AI, about being able to understand and manage AI agents to get work done as a software developer. The interview from Hard Fork Live covers the future of work and comfort in this new age. This reminds me of a book that the CEO of Redgate recommended, Reshuffle. I love the book, but it's slow reading as I constantly stop and think.
Work is changing; it's becoming unbundled and re-bundled in different ways, and many of us will have to learn to work in new ways. Not all of us, but many of us. Some might see their day-to-day efforts change little; some will not recognize their job a year from now. As with anything, lots of us will be in the middle with some changes, some status quo. That's certainly where I am with AI assistance.
Read the rest of Cognitive Coverage
Is it worth continuing to run SQL Server when PostgreSQL licensing is zero? Rebecca Lewis has a well written post on why that looks at some of the pros and cons of paying for SQL Server instead of moving to PostgreSQL. She starts with some of the things PostgreSQL does well, of which I think the Extensibility is really cool. SQL Server has some of this in CLR and the non-SQL language support, but those seem kludgy and complex to me. They aren't really integrated into the SQL Server platform.
They're good, but I do wish vendors or the community could add some extensions in a way PostgreSQL does. Of course, I also worry about stability, so maybe this is a wish that isn't really a great idea.
Read the rest of SQL Server Still Wins
I had a very interesting conversation recently with a longtime DBA who was worried about using AI in their database work. The Redgate State of the Database Landscape 2026 report showed that the vast majority of you (99%) are getting value from AI, so clearly it's being used. However, this individual was concerned that using AI for tasks would not engage their brain, and they might lose some of their SQL skills.
And they want to use their brain at work.
Read the rest of I Want to Use My Brain
I've visited a number of customers in the last few years who require most people to work in the office. Recently, I had the chance to go to Epic Systems, just outside Madison, WI, USA. They are a medical records software provider that was very reminiscent of Microsoft in some ways, and quite different in others. I published a blog with some pictures, so you can see how cool this office is in person.
Epic has all their employees coming into the main office every day. They are flexible if you have needs, but the expectation is that employees go in every day. I believe this is also their policy, and culture, in various offices around the world.
Read the rest of Spending Time in the Office
Last week we had a training session at Redgate Software on the Cloud. One of the first slides from John Q Martin asked the question, "what is the cloud?" The next slide had the answer: it's just someone else's computer.
I mean that's true, but it's not Grant's computer. He's got a creaky, 4 year old HP that I don't want running my workload.
Read the rest of What is the Cloud?
Change is inevitable for most of us. The jobs we hold, the places we work, the people we know, even our families grow and change over time. As I get older and live longer, I've learned to accept, appreciate, and flow with changes. I might resist, delay, embrace, or anticipate tomorrow, knowing there is always a positive and negative side to things.
This week, one of my colleagues retired. Annabel has been a part of Redgate nearly as long as I have, and we've worked together for many years. If you've ever attended a Redgate event, live or online, she likely had a part to play in the planning, organizing, execution, financing, and every other part of the process.
Read the rest of Changes, Happiness, and a Few Tears
For a while, I kept seeing that the cost of writing code was approaching zero. So many people felt that with an AI LLM, the costs would go way down to produce software. I'm not sure that's true. In fact, some companies are finding they spend more on AI tokens than salaries.
However, the ability to produce more code, experiment with ideas, or generate proof of concepts has gone up. Whether it's worth the cost or not depends on the engineer, but some organizations are finding that they can try more things than they would ever had time to try in the past. The time of engineers was the constraint, and if you can afford the cost, AI LLMs can relieve that time pressure.
Read the rest of Follow Your Hunch
Both as a DBA and developer, I've had plenty of immediate, this-is-broken, fix-it-quickly issues. Usually, I, or someone else, wrote some bad code and somehow got it deployed. I mean, I do test things, and I would (probably) never change code after I'd tested it to fix that one little annoying thing, like the formatting. I'd (almost) never do that, and I'm sure you wouldn't either.
Yet somehow bugs slip in at times.
Those are the acute issues, and they can be hard to fix at times, but often we can reproduce the problem in development and build a fix. Sometimes we even spot the issue quickly and just fix it in production. I'm sure you never do that, but I have had that experience myself a few times.
Read the rest of The Slow Growing Problems
Bjarne Stroustrup is the creator of C++. I read a few of his books and alternately loved what he'd done and hated having to write C++ code in university and at a few jobs. I found it tedious and hard, though arguably better than C once you had a decent set of classes structured. BTW, I love his website, the basic text view of the world, which is how I have built a few sites on my own.
I caught an interview with him and this short response on AI and coding. He had this quote: "Senior developers are already retiring rather than deal with it." He doesn't love the results from AI, which is fine. And it's not what I want to talk about today.
Read the rest of Would You Retire Rather Than ...
I ran across a statement that seems exciting to me as someone that has written a lot of code in their career. It said: "Many of the "modern" software practices of the last decade were early adaptations to this shift, even if we didn't articulate them that way. Immutable infrastructure. Stateless services. Containers. Blue-green deployments. Infrastructure as code. These ideas all share a common premise: never fix a running thing. Replace it."
These are a few sentences in this piece on the death and rebirth of programming. That's how a lot of software developers have viewed the world during the last decade and we've seen a lot of software advances in that time. The very successful developers and teams, who often speak at conferences and publish papers have adopted many of these practices. Serverless, containers, lots of tests allowing continuous deployment of new objects into complex environments that scale to levels many of us never thought possible. These are the very high performances talked about in the State of DevOps report every year.
Read the rest of The Data Model Matters
Lots of people move to the cloud; it's common. In fact, it's very common to hear customers who are being asked to migrate their workloads to a cloud vendor for a variety of reasons. You might not agree, but often there is some reason to move to the cloud. Sometimes it's even moving from one cloud to another, just because one of the big three (AWS, Azure, GCP) seems more attractive this year than the one from last year.
When you move, do you size your system for the peak? 80% of the peak? Perhaps there is another goal for which you design. Do you worry about ever being under-provisioned and letting customers have a slower system? Or do you ensure you never hit the peak, which increases costs?
Read the rest of Over or Under Provisioned
One of the things I used to emphasize in talks about DevOps is that no modern software of any significance is built by one person. Everything takes a team, so the foundation of version control becomes extremely important. We need a way to coordinate work across multiple individuals and communicate what changes are being made. This requires a strong foundation, and that starts with version control.
In 2026, that hasn't changed, but what has changed is the makeup of the team. No longer do I need a bunch of humans. In today's world, with extremely powerful AI LLMs, we can have a team of AI agents that write code, often at a pace far exceeding that of human teams. However, they still need to coordinate and communicate and ensure their changes mesh together.
Read the rest of The New Software Team
You still need DBAs (that know how to back up systems and test restores). If you think you don't, or if you manager does, then perhaps they ought to read this piece on how an AI agent deleted a production database. This wasn't the case of an agent just running around with sysadmin access to all resources, or a lack of tests that allowed bad code to flow through a CI/CD process.
This was a system design that had a hole in it. An API call to change infrastructure that could change both staging and production. Not something an AI set up, but humans did. A hole from both PocketOS and the API vendor that allowed the AI agent to make the same type of mistake we've seen humans make. A mistake of not double checking, not verifying, not following the rules of getting a second set of eyes, even a second set of virtual eyes, on the code that could drop resources.
Read the rest of Limit the Blast Radius
I wonder how many of you have tried vibe coding something with an AI tool. If you haven't, I certainly recommend it. I've been a bit amazed with a few of my AI Experiments, including my loading of a lot of inconsistently formatted data into a database for USD$5.
To be clear, there's plenty of vibe coding that might not be production-ready, but have you ever been handed code from a human developer you didn't think was production-ready? Or deployed code like that? Certainly, AI could exacerbate the situation, but it can also spark ideas, ease (and speed) development in small ways, and tackle the backlog of things your org needs.
Especially small tools.
Read the rest of What Can AI Really Do?
I remember getting started on SQL Server and trying to upskill myself in the mid-1990s. At that time, my employer was running a SQL Server 4.2 instance for a third-party application, but we wanted to rewrite our internal bespoke sales app to run on SQL Server. We were upgrading from Foxpro to Visual Foxpro and looking to move from shared dbf files to a SQL Server. There was a new release of SQL Server 6.5 during our development, and I wanted to learn more about it. I purchased Inside SQL Server 6.5 and read the entire thing, getting prepared to finish development and then manage a new platform in production.
I had updated copies of that book as SQL Server released new versions until SQL Server 2005. When that came out, there weren't one, but rather 4 books to cover the Inside SQL Server details (Programming, Query Tuning, T-SQL, and The Storage Engine). A similar thing happened with the SQL Server Bible, which grew in size to over 1400 pages for the 2012 version. It was a backache in a book if you put it in with your laptop.
Read the rest of There's Too Much to Learn
Many of us working with databases know the problems of a single point of failure. We build HA/DR technologies into a lot of systems precisely because many of us know if the database goes down, a lot of stuff goes down. Broken software is easier to fix and rollback, but a broken database can be a much bigger problems.
We also know an overloaded server doesn't handle a workload well, hence our quest for well-written SQL code, but we often lose that battle with developers.
Read the rest of The Dangers of Dependencies
While talking to a customer a few weeks ago, they mentioned that they used Contained Availability Groups (CAG) everywhere. They also said they were amazing and wondered why everyone wasn't using them in other environments. Of course, I questioned the "everywhere", which turned out to be more of a default for new systems than a standard across all systems. That's likely true of most things since it's rare we get to update/patch/set something across an environment of any size and ensure every system is the same.
Still, setting a CAG as a default makes some sense for enterprises. This ensures that in an HA situation I have my logins, jobs, etc. already on a secondary node. That's been one of the challenges of using lightly linked systems that only sync up database level information. Log shipping, Replication, Availability Groups can all work to keep a secondary ready to take over, but they all miss information that is stored in master or msdb.
Read the rest of Who is Using CAGs?
While working with a customer recently, I heard this sentence: a tool is better than a script. The reference was that this customer preferred a known, tested, approved tool for most of their staff rather than a script built, lightly tested, and perhaps changeable by anyone in their organization.
I was surprised, because in many ways, I've depended way more on scripts, more often, than "tools" in my career. Often I struggled to find tools that actually worked in the way I wanted them to and built them myself with Unix shell utilities, VB Script, PowerShell, or some combination of those or other technologies.
Read the rest of A Tool is Better than a Script
The AI LLM boom seems to show no sign of slowing down. Each time I think we've reached some level of crazy use or predictions, things take another turn. I still find myself pinging back and forth between this will be amazingly good and horrifyingly bad.
Sometimes on the same day.
Read the rest of Half of All Engineers
Recently I saw an interesting article, saying that someone could build a general purpose coding agent in 131 lines of Python code. That's a neat idea, though I'm not sure that this is better than just using Claude Code, especially as the agent still uses the online version of the Claude model from Anthropic to generate code or perform other tasks. There's a video in the article showing how this code can be used to perform some quick tasks on a computer.
However, the code isn't specific to Anthropic. It can be used with any LLM, and I started doing just that, with a copy of the code from the article, but modified to use a local AI LLM running under Ollama. You can see my repo and feel free to download and play with it. It's expecting a local LLM on 11434.
Read the rest of Local Agents
Every database platform has some strengths and weaknesses. Some more than others. I caught this site (NSFW) from Erik Darling, and it made my day. I was having a tough one when this site got me to smile and chuckle out loud a few times. I especially like the MySQL and SQL Lite links (again NSFW).
Every platform that you might choose to use to back an application can work in many situations. Certainly scale and load are factors to consider, but for the major relational database platforms, most will work fine for many applications. Some might work better than others, but there are always tradeoffs. There are pros and cons. This is also true for the major NoSQL platforms, though most of my experience is with relational ones, so I tend to lean in that direction.
At the same time, any platform can fail horribly.
Read the rest of Every Database Has Problems
In the last year I've seen a lot of statements about data and sovereignty between countries. While there have been concerns in the past, there seems to be more worry around the world with AI services primarily being run by, and hosted by, US companies. Plenty of my customers at Redgate Software have concerns over our ability to see data when we run AI models, though we don't store the data. Once the session ends, the data is discarded by policy
Recently I saw a piece about France trying to rid itself of the reliance on US technology, specifically the Windows OS from Microsoft. They are looking to move to their own version of Linux, as well as a number of open source software packages. This quote was fascinating to me: "We can no longer accept that our data, our infrastructure, and our strategic decisions depend on solutions whose rules, pricing, evolution, and risks we do not control."
Read the rest of The New OS Wars
One of my colleagues wrote a great post about DBAs and developers, about how a DBA's pushback on bad code isn't to be difficult, it's because they can see the future. I never thought of myself as a modern-day Nostradamus, predicting the future of system performance. Apparently I had another title besides DBA.
Working under pressure and with short deadlines often leads to short cuts. I've made them. I've implemented quick hot fixes. I've forgotten to port changes back to development databases. I've increased our tech debt load, just to solve a more immediate problem.
Read the rest of Working Better Under Pressure
There was a post on X recently from a founder in the EU about an engineer using Claude and ChatGPT to build a feature. I am not sure how true these posts are or if they are designed to just create engagement, but it's still an interesting topic. The part that makes me think is that (supposedly) the engineer was fired because their "data" (code) was sent to American servers. The code was then deleted and the feature will be built without AI.
First, read some of the responses before you form an opinion. There are some funny ones in there. There are a few I think are overblown and silly, and I skim past them. Someone is always more upset than I am, and more than I think they rationally should be, so I tend to let their outrage flow by me.
Read the rest of Who is Irresponsible?
It's always interesting to me when I give product feedback to engineers at Redgate on their demos. Quite often they've built a feature that uses AdventureWorks or Pagila (PostgreSQL) or some other well known schema to evaluate how their particular thing works with a database. I try to remind them that many databases aren't well modeled and designed with consistent naming.
I ran across a Daily WTF article that isn't showcasing databases, but it does show some poor naming in data being stored in a PDF. The developer who had to automate a process had to map these fields to database fields, which also might not be named very clearly. In fact, I think I've seen a few database models that used column names like the field names in the PDF.
Read the rest of Poor Names
Recently, I saw a graph about making decisions that showed the impact of both reversibility and consequences. Here is an example of such a graph and how one might approach decisions. If things are easily reversible or have a low consequence, we tend to make a decision and move on. Or we are willing to make a decision. One of the examples of such a decision was choosing what to wear out to dinner. It's easy to change, and (in general) of little consequence. Choosing to send a large amount of money to someone through Venmo (or some other mechanism), can be hard to reverse and have substantial consequences.
This made me think of some of the DBA and developer decisions I've made in the past. When we work with databases, the changes we make can have a large impact and be quite consequential to our organization. Downtime, data quality, etc. could all impact revenue, profit, reputation, or even future prospects of survival. That can be a lot of pressure when you are deciding to refactor a data model or adjust a lot of data during a deployment.
Read the rest of Acting with Confidence
Years ago I was giving a talk on software development and asked the audience how long it takes to review a PR that has 10 lines changed. Answers were in the minutes to tens of minutes range. I then asked how long it takes to review a PR that has 1,000 lines changed. Some people said hours, but a few people said seconds.
I've often taken the latter, pessimistic view. Not because I don't think engineers want to do a good job, but because I know human behavior. Most humans will get bored, lose focus, and end up skimming through a large amount of code. Many (most?) people don't want to spend all that time, after all they have they their own code to write. They'll just approve the PR and assume testing will catch any major issues.
Read the rest of Barely Reviewed Code
SQL Server Central has been a great success over the last 25 years. We've helped a lot of people improve their careers with the Microsoft Data Platform, primarily SQL Server, but we've published articles on other aspects of databases, including other platforms. I wrote a bit about the history of the site last month, with a few stories in various pieces. We even got Brian Knight to contribute a piece on what the site meant to him.
Over the years, we experimented with trying to get an SSIS Central or a SSRS Central off the ground. However, we struggled to find other people who would have been willing to partner with us to provide content and answer questions. Eventually, we gave up, though I wish today we'd have pushed forward with a PostgreSQL Central site a few years back.
Read the rest of AI Database Central
One of the challenges of AI-assisted coding agents is that they tend to produce A LOT of code. Even in refactoring or migration changes, the AIs can work quickly and generate such a volume of code that the process starts to become overwhelming. For pull requests, for CI/CD build systems, and certainly for human reviewers, they can be overwhelmed. This can become a real problem with OSS projects, where submissions can grow exponentially to the point that maintainers stop looking at pull requests. I suspect the same thing might happen in corporate repositories when lots of developers can refactor or submit huge amounts of code produced by AI agents in a fraction of the time it took a year ago.
I was listening to an interview with an experienced software developer and OSS project maintainer who said that he preferred getting a "prompt request" that contained a description of a problem and the specification for a solution that he could submit to his own LLM to get the code. Rather than use an AI to review a code in a PR written by a human or AI agent, a great prompt that can communicates the problem and solution is preferred.
Read the rest of Prompt Requests
We do a lot of research at Redgate Software, often contacting customers or Ambassadors and asking questions, getting feedback, listening to their challenges or criticisms. We take that information and it helps guide us to build the tools that help you in your job.
Recently I watched a call with a customer that uses our famous Toolbelt in their job and has for years. In this case, the call was about the value of Flyway and how that might fit in their organization. The person performed a lot of manual work to deploy code changes, and could see the way in which Flyway helps, but wasn't interested in saving time.
They saw their time as free. I assume their employer felt that way as well.
Read the rest of Is Your Time "Free"?
I wrote a blog about sitting at LHR recently and watching planes take off. That's been a fun thing for me to do when I'm stuck at the airport. I can see a plane roll down the runway every 35-45s during busy times. This time I was sitting by a window in the hotel, working and watching.
There was a moment when I realized no planes were taking off. I looked and saw a vehicle rolling down the runway, and then realized there were two, one from each direction. It was an runway FOD inspection, looking for anything that might damage a plane.
Read the rest of Doing the Little Things
When I was young and needed to learn about something, I had to go to a library or a bookstore to get information. I often started by looking through an encyclopedia. I had to wander between entries to learn more about the topic I was researching. A few lucky friends had their own copy of an encyclopedia, which was nice since we could research at home. At some point these collections of information were moved to CD/DVDs, which made them more portable and more accessible to a larger group of people than in the past.
In the 90s we had the innovation of search engines, which allows us to more quickly move through information. There was more information available on the Internet than was ever published in encyclopedias. Over the years, these search engines improved their ability to rank and recommend information that is relevant to your query. However, you still need some idea of what you are trying to learn about. You have to direct the searches, although the Google auto-complete felt very predictive at times.
Read the rest of Everything is the right question away
A few weeks ago I was in Bletchley Park, at the facility where the Allies decoded and broke many of the German Nazi messages in World War II. It's quite a facility and museum, and I hope to go back. I was distracted that day and didn't get a lot of time to enjoy the exhibits and really learn more about what happened there.
I was there for our Redgate 2026 Company Kickoff, and as a part of that, two different executives in our company shared their stories of people who had worked there. What was interesting is that until we planned this event, these two people had no idea that there were people they knew well, who had been part of the effort to end World War II at Bletchley Park. This facility can be considered to be one of the birthplaces of computing.
Read the rest of Hidden Heroes.
Recently I was working with a customer that is trying to smooth out their database development process. They noted that a lot of changes tend to take a significant amount of time and impact their availability. They came to Redgate to see if Flyway might be a way to improve the effectiveness of their internal groups that build, deploy, and manage their database software.
We can help, but often when I get called to help architect things, we are trying to decide if the customer is a good fit for our solutions. Since we work on a subscription model, we want to be sure customers are happy and get value from their decision. Otherwise they have a poor experience and don't renew. This might be because they aren't ready, or it might be that the question wasn't considered of whether our solution fits their environment well. In any case, I usually dig into the goals and challenges they've faced with their current process.
Read the rest of Changing Data Types
I saw an interesting LinkedIn post on Kyler Murray and how he goes about approaching the game of American football. I don't know if this meme is true, but certainly, his efforts to prepare have been a reported issue during Murray's career. The post actually deals with sales and analyzing the reasons for deal success or failure, something I've been able to witness at Redgate the last few years. It's interesting to me to see the sales process examined, though I don't make sales.
Incidentally, one of the comments is one I appreciate, referencing Kobe Bryant and the Mamba Mentality. I like the approach of working and asking questions to become better.
Read the rest of Breaking Down Your Work
We had a Simple Talks podcast recently where we discussed roll forward vs roll back. You can watch the episode and listen to our thoughts, but one interesting place was when we talked about deployments. Grant mentioned that he deployed from version control/source control at a previous employer. I asked him whether he did that for every system.
His response: "Well, ..."
He admitted that most, but not all, databases came from a controlled source. There were some systems that had a more ad hoc change process. I wonder how many of you have consistent processes throughout your organization. I suspect not many of you do, especially if an organization isn't small. Often, different groups and applications are in a constant state of flux, with lots of different processes and protocols.
Read the rest of Multiple Deployment Processes
I have the opportunity to work with a variety of customers on their database systems, often with the focus on how they can build and deploy changes to their databases. Often, they have a process around how and when they make changes. Some have maintenance windows, though often these are approved times for changes rather than a true window during which a system is shut down.
I ran into a customer recently who scheduled a system shutdown for their deployments. This was a surprise to me in 2026, as I thought most people would have learned to deploy changes to live systems. However, I know that many teams make changes that would render portions of the database inaccessible for a period of time, so maybe that's not true. Maybe they just make changes and deal with the impact on clients.
Read the rest of A Full Shutdown
I remember listening to an interview with Rick Reilly in the mid 2000s. He was the back page columnist for Sports Illustrated for years as well as a writer in various pieces. He talked about how he would lay on the couch in his office sometimes, trying to think of what to write. His kids would come in looking for attention, but couldn't understand that Dad was "working".
I had been writing the editorials at SQL Server Central and I could relate. Moving from 2 to 5 (eventually 6) editorials a week was a lot of work. It was stressful in a way I couldn't imagine when I started writing them. I quickly realized that if I had to produce a new one every day, I was in trouble. There would be days I'd struggle. I needed to have a queue of pieces at least partially ready if I were going to manage this job and find balance with my family.
Read the rest of Writing as an Art and a Job
The Advocates at Redgate Software had an interesting discussion about deployments in databases and how you go forward or back from the point at which you discover a problem. You can watch the episode, but a few things occurred to me while we were having our discussion.
The first thing is we all agree data makes things hard. A database is a stateful object, and dealing with stateful objects is hard. That is one of the things I've internalized the last few years that has tremendously changed how I work with Redgate customers. The more I consider state, the more I am able to work with the challenges that databases bring.
Read the rest of Rollback vs. Roll Forward
Many of us know that testing our code is important. The adoption of unit testing by many software application developers as a normal course of business has dramatically improved the quality of applications. Mobile software, especially, has benefited from the requirement for most software to include, and constantly run, a suite of unit tests.
For database software, I find relatively few organizations formally test their database code. A few people have adopted tSQLt or the Microsoft Unit Testing Framework, but most don't bother. In fact, many queries that are embedded in application code, or built by ORMs, aren't tested beyond a developer looking at the results from their own (limited set of) test data. That often doesn't catch errors until someone in production runs their application against a larger set of data.
Read the rest of Testing is Becoming More Important
Why do we reboot machines when something goes wrong?
I'm sure all have done it, and I would guess quite a few of you have found situations where this seems to fix issues, but there isn't an underlying root cause that you can pinpoint. This is a fairly accepted way of dealing with issues, but have you thought about why this is a way to solve some problems?
The main thing that a reboot does is return the system to a know starting state. It's why quite a few people complain about some modern laptops and mobile devices because they avoid restarts and try to sleep/wake instead. Most software expects to work on a stateless machine, so restarts help find a known good state.
Read the rest of Can You Let Go of Determinism
This is part of a few memories from the founders of SQL Server Central, celebrating 25 years of operation this month.
We did photoshoots at Redgate many years ago. We had a bunch of props, including some phrases written down. We could create our own, but my handwriting is atrocious (likely why I never became an architect), but I ended up with this one:
Read the rest of Doing Good at SQL Server Central
Many of you reading this have a number of years working with technology. You might have 1 year or 20 years, but you've likely grown and learned along the way. Some of you may also know someone who has several years of seniority in a position but not that many years of experience. In this case, a person might have been working at this job for 5 years, but they really have one year of experience that's been repeated 5 times.
That's been a common complaint over quite a few years from people who interview others. They find candidates often have very limited experience, yet are applying for senior roles. These candidates are ones who have just a few years of experience, but have ended up repeating those few years over and over.
Read the rest of Engineer Lessons
This is part of a few memories from the founders of SQL Server Central, celebrating 25 years of operation this month.
When we started SQL Server Central, our goal was to build a great resource that helped other people advance in their careers and also made some money. Our decisions in building the site were based around the digital world and treating the community as we would want to be treated. Over time, however, we realized that continuing to grow this business was hard in a digital-only world. We experimented and proposed helping others build similar sites, like ReportingServicesCentral (which would have been great) or NotificationServicesCentral (which would not), but ultimately, we weren't experts enough in those areas and couldn't find people willing to partner.
Everyone thought they could do it themselves and that the knowledge was the hard part, and execution was easy. The truth is that the reverse is the way it works.
Read the rest of Expanding into Print
I tend to be fairly careful with data, especially data on this site. When we started the site, we were worried about potential issues and worked hard to ensure we kept our systems safe and limited the attack surface area for personal information. We also declined the various offers we had to sell our list of subscribers to marketing firms. We know that some places add value for marketing, but some abuse the trust of their users and our approach was always to be careful.
When we sold the site to Redgate, we emphasized the need for this trust, and to date, Redgate has been a great steward of your personal information. I regularly field requests for uses of data from other marketing people, and almost all get declined. I've had a number of great managers who have supported me on this because we value your privacy.
Read the rest of The Power of Data and Privacy
I remember getting a job at a startup in the Denver Tech Center. This was shortly after SQL Server 7 was released, with a marketing campaign that the platform was auto-tuning and wouldn't require a DBA. My colleague asked me if I wanted to learn Cold Fusion and have a longer career. I declined and stuck with this SQL Server thing, which has seemed to work out pretty well over the years.
I was reminded of this when I saw a "Death of the DBA Again" post, this time from an Oracle DBA. There are plenty of links in there from Larry Ellison and Oracle about how some version of Oracle won't require a DBA. I've seen questions on Reddit (and elsewhere ) about this topic where people seem to think DBAs can be replaced.
Or maybe they want them replaced.
Read the rest of The DBA is Dead; Long Live the DBA
There have been a lot of features added to the SQL Server platform over the years. Several of these features let us perform functions that are beyond what a database has traditionally been designed to handle. SQL Server has had the ability to send emails, execute Python/R/etc. code, and in SQL Server 2025, we can call REST endpoints.
Quite a few of these features (arguably) are more application-oriented than database-oriented. There's nothing inherently wrong with having a server perform some of these functions, and there have been some very creative implementations using these features. I recently ran into one of these examples from Amy Abel, where she shows how to use the new REST endpoint feature to call an AI LLM to generate and send emails from your database server. That's creative, and it's reminiscent of the numerous examples from various experts over the years who demonstrate how these features can be used to accomplish a task.
Read the rest of Expensive CPUs
The oldest article we have on the site is Tame Those Strings! Part 4 - Numeric Conversions, by me. It's dated 2001-04-18, though I think that's a date we picked when we converted all the content from one database to another. The founders agreed sometime during Feb 2001 to jointly run SQL Server Central. Since we each owned the copyright of our articles from another site, we migrated several articles to build up our content library. This was back when Andy, Brian, and I all had full-time jobs and managed the site during breaks, nights, and weekends.
That was 25 years ago.
Twenty. Five. Years.
Read the rest of 25 Years of SQL Server Central
I was reading Andy Pavlo's end-of-year review of the database world. He's done this for a number of years, and there are links to previous recaps in the piece. He is an associate computer science professor at Carnegie Mellon University, working on quite a few database-related projects. In the review, he tends to track the database world from the perspective of business success and money. There are certainly parts of it that discuss technical changes, but my overall impression is more about the business and usage success than it is about the way database systems work.
The main thing that struck me after reading the review was how many database systems there are in the world. I hadn't heard of any of these: RaptorDB, TigerData, Tembo, StormDB, Translattice, FerretDB, DocDB, SpiralDB, Tantivy, SkySQL, HeavyDB, and more. I'm sure I missed listing some I didn't recognize, and quite a few of these are PostgreSQL-based systems, but still, that's a lot of database systems that exist and are having success.
Read the rest of There Are a Lot of Databases
AI is everywhere, and if you spend any amount of time looking for answers on the Internet to your coding challenges, you've likely encountered a lot of poor, average, good, bad, amazing, and just-helpful-enough AI content. For awhile, I was avoiding the AI summary from Google as the quality seemed slightly off, but lately it's gotten good enough that I tend use it to decide which links to click on in the results. The summary helps me better understand the context Google sees in my search query.
I ran across a post on coding documentation and how helpful these docs are in onboarding, code reviews, and more. The teams that worked smoothly together often had good docs that helped them function as a cohesive group. At least to some extent. Over time, teams start to depend on tools and lose some of that cohesiveness since they rely more on tools than docs. I agree with the piece that this is a part of the reason many teams don't really function as teams over time.
Read the rest of More Documentation is Needed
There's concern about the future of AI and how it may affect jobs and employment for the masses. I see plenty of people on both sides of the issue. Some are sure AI technologies won't replace people; some are concerned their jobs will be eliminated, and some are hoping that we will eliminate some jobs and create many more.
Sometimes that's the same person.
Read the rest of Deep Learning and Craftsmanship Matter
I've had the fortunate, or maybe unfortunate, experience of being thrown into a few jobs with no training. At a couple of my bartending jobs, I had to start working without any training, calling over someone to help run the ordering machine while I made and served drinks. I managed to slowly learn how things worked throughout that first shift, so I was ready to work on my own the second night. I had a similar experience at a tech job, starting as the lead DBA/IT Manager in a crisis, having to try and solve problems after ask others how things were supposed to work. I ended up fixing a bit of code, adjusting networking, and directing others on my first day.
When we have a crisis, we often learn a lot from the situation. I've been through crashed upgrades, virus breakouts, hardware failures, and more in my career. While each was stressful and often not enjoyable, I learned a lot each time and came through the incident a more capable developer/DBA/whatever. When we work through a tough time, we are often better equipped for the next time something goes wrong.
Read the rest of Learning From Breakage
When I was at the Small Data 2025 conference, one of the speakers was talking about their work with AI technologies. This person uses it a lot in their day job, often to complete tasks that they would have struggled to work on in the past, mostly because of time constraints, but also a lack of resources. Sometimes this person has an idea, but doesn't want to distract themselves or others by having them work on a side project.
During a recent ride in a Waymo (self-driving car), this person had their laptop out and running Claude Code. They gave it a prompt, asking it to build a small app for some data analysis. During the 8-minute ride, the agent had spit out the code, a Readme, and committed this to a git repo. Later, the speaker tried it and found it solved most of his requirements, and then did some other work on the project, as well as having Claude write more code to get something that was beyond a minimally viable app.
Read the rest of Eight Minutes
JSON seems to be everywhere these days. Many application developers like it across all sorts of languages, C#, JAVA, Python, and more. They use it for transferring information between systems, and are comfortable serializing hierarchical object data into JSON from text and de-serializing it back into its various elements.
For those of us working in relational databases, JSON seems like a blob of information that isn't easily queried, indexed, or stored. We prefer working with a relational set of data, which brings us into conflict with software developers. We'd like them to convert their objects to a relational structure, and they'd like us to just work with JSON.
Read the rest of JSON Has a Cost
I came across a post recently on the Microsoft Fabric blog about the evolution of SSIS 2025..I hadn't heard much about SSIS in SQL Server 2025, so I thought this might provide some info on the investments that Microsoft is still making in Integration Services. I've run into a few people in the past year who are still heavily invested in SSIS and run packages daily. SSIS seems to be a technology that isn't even close to dying for many organizations.
The blog starts well, delving into the security investments with the change to the SqlClient and TLS 1.3, as well as supporting Strict Encryption. I don't know many people using this level of security, but it's good to have SSIS support stronger security. There is also an upgrade for SSIS packages targeting Fabric Data Warehouses if they modify their approach.
Read the rest of An SSIS Upgrade
I ran across a post that discusses what makes you a senior engineer (via Brent Ozar). The main point of the post is that there is a core skill that separates senior engineers from others, which is reducing ambiguity. When a senior engineer gets an ill-defined (or ill-communicated) request, they can deliver a solid, or even great, result.
When someone says "performance is poor," what do you do with that? Can you build a plan to identify the issues and solve them? Or do you expect the customer to explain what is slow and why it's slow? What metrics do they have showing things are slow? A senior engineer can ask questions to find the problem and then determine how to move forward.
Read the rest of Where Your Value Separates You from Others
Recently, I was discussing AI with a friend, and they asked me to name a great success of using AI to build software. I've tried a few things, and I've worked with customers who are using AI tech. However, most of the things I've seen built with AI are small tasks; they're utilities or quick wins that change a minor part of the software. The items tend to be tactical and focused in a narrow band of fixes, and they might save a programmer time, but I'm not seeing large-scale team improvements in productivity.
Yet.
Read the rest of Your AI Successes
Security has been a constant concern for many IT professionals over the years. Many of us are trying to implement better security controls, and yet at the same time, we try to avoid anything that slows us down. Security clearly hasn't been a big enough concern, as we've had more than our share of SQL Injection issues. These often come about from poor practices, lack of education, and too many people not learning to adopt better habits across time.
We've also had no shortage of lost backups, open cloud buckets, and more over the years. While security (or cybersecurity) is listed as a concern for tech management, they are quick to avoid slowing down any development or deployment of software. While it is easier to get time for patching these days, it's still not easy. There are plenty of organizations that prioritize resources spent on tasks other than patching, upgrading systems, or training developers.
Read the rest of Minimally Viable Security
It's the beginning of the year, and some of you likely have today off. But plenty of you are at work, moving slowly through this Friday at the start of the year—handling busywork, catching up on maintenance you've let slide, or preparing for the tasks you know will start coming Monday.
At Redgate, most engineering teams work toward a North Star goal: a high-level direction that guides your various tasks. Perhaps it's growing a customer base or achieving an overarching product specification. For example (this is completely made up), one North Star might be achieving feature parity across all platforms for SQL Compare.
Read the rest of The North Star for the Year
I ran across a tweet (are they still tweets?) on X/Twitter that was titled: how to ruin yourself. It had these items, which seem to be coming from a young person. Either a student or in their first job.
- Stay on your phone all day.
- Feel sad for no clear reason.
- Stop eating well and ignore your studies.
- Sleep super late and wake up in the afternoon.
- Let sadness take over everything.
- Always look at others' lives and feel yours isn't enough.
- Keep blaming yourself for the past but never try to let it go.
- Compare your progress with people who started years before you.
- Get stuck imagining outcomes instead of creating them.
- Keep waiting for motivation instead of building discipline.
What was interesting to me is I saw people doing similar things when I was younger. Either adults with careers or fellow students. I'd change "sad" to "anger", which I saw a lot in the 80s. Replace the phone with TV, as I saw lots of people start to invest a lot of time in TV with the growth of cable and 24-hour channels in the early 80s. Eating well was less of a thing, but drinking more was a thing. However, many people stagnated, or maybe ruined, themselves in similar ways.
Read the rest of Finding Motivation
Most of you reading this are likely technology professionals of some sort. You might be a software developer in C# or a DBA or a manager of those teams. Maybe you're an analyst working with data and reporting. You have made this a career choice and (hopefully) are growing and learning more about your craft.
I also expect that you want to continue working in the area you are now, or maybe want to move into a related area. Maybe a report writer wants to move into more warehousing/lake housing. Maybe a DBA wants to be a Reliability Engineer. You have a career and you're working in that area.
Read the rest of The Side Job
The PASS Data Community Summit 2025 was held in Seattle last month, and it was an interesting event for me. I wrote a wrap-up on my blog, but a few things stood out. The event was a little smaller, with over 50% first-time attendees, but seemed to be a bit more vibrant. Perhaps people coming for the first time added something that I hadn't expected. I was a bit over-committed, so I didn't spend a lot of time in the public spaces, but things felt a little different the few times I was in the expo hall or the hallway track.
I ran across a Reddit thread on the value of conferences, and it got me thinking. What is the value that you get from attending a conference (or an event). If your employer pays you might feel that you should bring some value back to them when you return. That's the premise of the thread, and I know there are plenty of people that feel that way. However.
Should you value your time and effort any less?
Read the rest of Your Value from a Conference
In his book, The Coming Wave, the CEO of Microsoft AI laid out the risks of AI tech bluntly. "These tools will only temporarily augment human intelligence. They will make us smarter and more efficient for a time, and will unlock enormous amounts of economic growth, but they are fundamentally labor-replacing," he wrote. Suleyman advocated for regulatory oversight and other government interventions, such as new taxes on autonomous systems and a universal basic income to prevent a socioeconomic collapse. This book was published before Suleyman joined Microsoft.
Satya Nadella is more optimistic than his new deputy. In an interview at Microsoft headquarters, while sitting next to his human chief of staff, Nadella said that his Copilot assistants wouldn't replace his human assistant. As his chief of staff sat typing notes of the conversation on her tablet, Nadella acknowledged that AI will cause "hard displacement and changes in labor pools," including for Microsoft. Judson Althoff, Chief Commercial Officer, said that Nadella was pressuring his team to find ways to use AI to increase revenue without adding headcount.
Read the rest of The Challenge of AI
One of the things I see software developers often talking about is how they refactor code. As they touch a class, method, etc., they may take the time to refactor the code to make it cleaner, perform better, or just add some documentation. It seems that a regular part of a software developer's job is refactoring code in the codebase.
That is unless they see a "don't touch this, no idea how it works" comment. There are plenty of those, and often everyone leaves that code alone.
Read the rest of Refactoring SQL Code
The GenAI boom is growing like crazy. From hype to disasters to successes to investment to the embedding of GenAI tech into lots of products, it seems no one gets away from AI. My wife, kids, friends, they all talk about AI and alternately give me stories of huge successes or epic failures. Even those who just scroll through reels aren't immune as we see amazing things, but we can't trust them because of AI. Who knows what image/video/audio was actually recorded and what was generated.
Like many of you, I think AI can be amazing. Like more of you, I think it can be a really poor partner and it produces output I can't trust. I think one of the major challenges is learning to treat an AI like a colleague whose work quality is erratic. It's not that I can't work with them and use their work, but I need to test, validate, and verify the code they give me does what I need, at some acceptable quality level.
Read the rest of Investing for AI
Recently I saw an article on Simple Talk, 15 Practical Tips for Securing SQL Server, and I thought that many of these are fairly simple things. Turn off unused features, disable sa, etc. These are things that a lot of people probably ensure are in their SQL Servers builds.
Though, I'm sure a lot of people don't bother.
Read the rest of Your Security Checkup
Imagine a perfect world? I have an AI agent that knows my business well. It's getting real time input from sales, from customers, it makes amazing decisions. We get a large order? We need to ramp up production of our widgets. We have an order pipeline of xx widgets and we know over time that yy% will close. Let's place a larger order with a supplier overseas.
The next day, we have an election and tariffs are announced on imported parts. We react immediately, cancel the order, start the process to expand a local factory. We place ads to hire workers and order equipment. Things are looking good for our business and our factory will be up and running in a few months.
Read the rest of How Important Are Real Time Decisions?
Over the years I've had no shortage of licensing questions for SQL Server. At times it's felt a little crazy. Look at the licensing guide. Choose EE or SE and the number of cores. Then check if you're using VMs. Oh, and consider the cloud, and which cloud you're running a workload on.
It's simple right?
It can seem confusing, and at times I've wished Microsoft would make it simpler. And perhaps even give us some add-ons, like adding some additional hardware capabilities (cough more RAM *cough) in SE.
Read the rest of SQL Server Licensing is Simple
If you graph computer/query cost against the size of data, you can get four quadrants:
- small data, small compute (most CRUD app queries)
- small data, big compute (complex BI queries for this quarter, most reporting)
- big data, small compute (logs, audit data)
- big data, big compute (complex BI queries across all our data)
Read the rest of Don't Let Corner Cases Drive Your Design
A few weeks ago, I was at the Small Data SF 2025 Conference in San Francisco. I attended the inaugural event last year and decided to go back again. It's a great chance to hear people thinking about data and its impact on the world in a different way, recognizing that building lager and larger systems isn't always possible. Or a good idea. We might find that smaller systems fit well, especially smaller datasets, which can both serve our purposes and create agility. The manifesto of the conference says that "We champion the power of Small Data and smart AI, believing that less is truly more." There's a bit more, but that's the idea.
The format for the conference is a little different, with 3-5 talks in a row, all on one stage, each about 25 minutes long. These are talks with or without slides, but no live demos, just speaking and expressing a point of view. What I found fun was that each person picked their own music to play as they walked onto stage (or ran/danced in the case of Glauber from Turso). It was a bit of fun, with the DJ letting the music play as the person made their way to the front and were welcomed by the audience. I heard rock, metal, hip hop, and more.
Read the rest of What's Your Theme Music?
Mary Spender is a musician in the UK who I follow and hope to see live one day. She works hard producing content about music, that business, and, of course, songs. Recently she had a little essay on Instagram where talked about creative time and focus. In it she referenced Elizabeth Gilbert saying "done is better than good."
My initial reaction was "that's right."
Read the rest of Done is Better than Good
Recently, I had a few questions on database modeling. One was posted in the SQL Server Central forums, and a customer asked about ERD tooling on the same day. This came shortly after Redgate acquired Vertabelo (now Redgate Data Modeler). This stood out to me as very rarely in the last few years have I found people consulting and updating a diagram while performing database development.
When I started as a developer and needed to update a database, I had to first update a diagram that was stored in ErWin. We had a dedicated computer (back when we went to an office every day) where the software was run and any developer could us this to update the diagram with proposed changes. Back then, we had to get another peer to sign off on changes before making them, and the peer was supposed to go check the diagram for the change before approving it. That's only if they thought your change made sense and conformed to our standards (naming, design, etc.).
Read the rest of Is Data Modeling Common?
SQL Server 2025 was released this week. The announcement came at Ignite and the PASS Data Community Summit with keynotes on Wednesday and Thursday, respectively. While there are some things to look forward to in the release (What's New) and some highlights from T-SQL Tuesday this week, it seems that this release wasn't a very exciting one.
On one hand, I blame all the Microsoft Fabric focus, which seems to distract from the core product that powers the databases at many organizations. The SQL Server blog from Microsoft has had relatively few posts this year, highlighting a few things. The Fabric blog gets more posts, which is something I've seen at Database Weekly as well. As I curate the content during my week, I find a lot more Fabric-focused content than SQL Server-specific posts. That contributes to a lack of excitement for a new version of SQL Server.
Read the reset of An Unexciting Exciting Release
I remember a time before email. Some of my first jobs were mostly based on paper being moved from person to person. I'm sure some of you remember these envelopes being used to communicate between individuals in an organization. I used those to send and get memorandums from others before we implemented email. Fortunately, our email implementation (cc:Mail) came soon after I started working in corporations.
Initially, people treated email much like paper mail inside organizations. However, over time, people started to treat email differently. It was easy to send an email around other work, so people started to send more messages than they ever would have with paper. They started to dash off notes quickly, sometimes too quickly, as an email might be followed by another email that includes a "I forgot this". As instant messaging grew, we saw similar patterns where people were quick to send messages, regardless of whether they were important, well-thought-out, or even necessary.
Read the rest of Don't Create Workslop
Over the last few years, I've worked a lot with various customers on finding better ways to build database software, often using the principles of DevOps to drive the change. A lot of managers and leads want to see a smoother process to help their teams become more efficient. DBAs often want less overhead and friction in the process, while developers just want to deliver code.
In many cases, however, what lots of management wants is speed, and they're looking for ways to increase their current speed and deliver more software. Their current rate of development might be quick enough if you can reduce your bottlenecks. Making communication easier, limiting the slowdowns from handoffs, and reducing the risk of mistakes are everyone's goals.
However.
Read the rest of Being Mindful of Design Time
Imagine that you are about to tackle a new project that will take more than a year. This might be a new system, perhaps a cloud migration, or maybe it's rewriting something that doesn't work well. You don't have enough employees to undertake the project without overloading everyone. Your team needs to grow.
Would you rather hire a more senior person from outside the organization or pick a junior person that's already inside your company and teach them what they need to know? Think about this as if you were the one making the decision about the future direction of your software team. Philosophically, do you want to buy experienced people or train/build new ones from your internal staff.
Read the rest of Internal Staff Growth
I ran across a thought-provoking post from Chrissy LeMaire asking if we should reconsider SQL Server HA. The post actually asks if you've considered not using it. The default from Chrissy, for most installations, is to use standalone SQL Servers. This isn't to say she's against HA solutions (FCIs or AGs), but that they often cause problems and might not be needed.
It's an interesting position to consider. For a long time, I avoided SQL Server clusters as they were hard to setup with a lot of complexity, hardware requirements, etc., and didn't really provide enough benefits over using log shipping with a second server for me. These days I have clients with mostly AGs, and they seem to run fine. That being said, Chrissy notes that after she left a job, a network outages caused a bunch of downtime. I could see there being downtime, as the old database mirroring (and the it-will-never-die replication) needed a working network. If you have network issues, you better know how to manage your HA technology's issues.
Read the rest of Do You Really Need HA?
I wrote recently about some work with Redgate Clone, and one of the things I did was start up a blank container instance of SQL Server from the image named empty-sql-current. This image contains SQL Server 2019. Clearly, "current" was a poor choice.
I see this often in various places, where someone will reference "current", "new", "latest", or some other term that denotes the most recent changes. If everyone reading the reference is doing so with knowledge of the past and at a time close to publication, this works fine. However, a year later, does this make sense? At the same time, I do like consistent names that might be used in scripts. If I always want developers pulling the latest item, I might use latest. However, if versions are important, than "latest" or "current" might not be the best choice. Much of the time, I tend to try and get a version or some other specific indicator in a name.
Read the rest of Poor Name Choice
There is a ton of hype now about using GenAI for various tasks, especially for technical workers. There are lots of executives who would like to use AI to reduce their cost of labor, whether that's getting more out of their existing staff or perhaps even reducing staff. Salesforce famously noted they weren't hiring software engineers in 2025. I'm not sure they let engineers go, but it seems they did let support people go.
For many technical people, we know the hype of a GenAI agent writing code is just that: hype. The agents can't do the same job that humans do, at least not for some humans. We still need humans to prompt the AIs, make decisions, and maybe most importantly, stop the agents when they're off track. I'm not sure anyone other than a trained software engineer can do that well.
Read the rest of Data > Hype
I wrote recently about a bad first day for an intern. He/she was fired, without cause in my opinion, when a production database was damaged while following a document for developer setup. The situation felt like a mistake, and one that wasn't necessarily the fault of the individual. To me, this was extremely poor handling of the situation from a CTO.
In the discussion for the piece, someone pointed out that it might not just be a new employee who makes a mistake that causes downtime. Certainly, an inexperienced employee could have caused the issue, but I know there are plenty people with lots of time in a position who make similar mistakes. It could be that one who has been there a long time followed a poorly documented procedure for the first time, or applied the procedure to the wrong situation. Often, I find these are relatively simple mistakes because someone isn't as familiar with a protocol or skill as another person assumed they were.
Read the rest of Practice Until You Don't Get It Wrong
I ran across this article on a survey about AI usage recently. The headline is this: 55% of businesses admit wrong decisions in making employees redundant when bringing AI into the workforce.
That sounds a little ominous for those making these decisions, and a lot of you might be saying, "I could have told you that. Using AI to replace people is a bad decision." On the surface, I agree. I dislike the idea that companies will opt for a semi-competent AI bot or agent to replace people, thereby further exacerbating the challenges faced by many workers in the modern world.
Read the rest of The Selfish Case for Learning AI
Cloud costs are high and growing. Some orgs think they're out of control and are trying to limit spend. Some orgs are looking to leave the cloud. A lot of IT spend over the years has been seen as a cost center, with many executives trying to limit the growth or spend, even while they aim for digital transformations of their businesses. Throughout my career, it's been interesting seeing the tension of groups trying to take advantage of technology and the finance departments trying to manage costs.
The cloud brings some of the same debates/arguments/concerns to the forefront. Partially because of scale, as we can add cloud resources much quicker than we can with a CapEx purchase. Partially because we've also often lost some control over budgeting with the move to OpEx and subscription things.
Read the rest of Reducing Cloud Cost
Recently, I got a bill from Azure. That's not an unusual thing for many of you, but for me it was a surprise because it said I was late paying. I've had a number of services running, and I thought at first that I had left something running too long, like a VM. As I checked, most of the things were paused, even the expensive ones like a Synapse workspace. Instead, I found that my free credits were not being applied. Fortunately, I had changed credit cards or I might have been billed for a few months before I noticed.
This was a change in how Microsoft managed benefits, which is fine. I opened a support call and someone helped me, but it took several days to get a response. I was slightly worried about the bills, so I decided to audit the things I had running.
Read the rest of Cleaning Up the Cloud
I've spent quite a bit of my career as a DBA/sysadmin/Operations person. However, I've had my share of development positions as well. As I work with customers who look to mature their database development to be more like other software development, I've noticed that PRs sometimes don't get handled as smoothly as we might like. In some sense, they are like help desk trouble tickets that never get closed.
One of the first things I caution people about is specifying specific reviewers, especially DBAs. There are often DBAs who are the gatekeepers for code, but if we require them to be the only ones to review code before a CI or test process, we really slow things down. This often happens in smaller environments where one DBA wants to avoid anything impacting their job. They want to review everything before it commits.
Read the rest of PRs Are Like Trouble Tickets
Some of you out there are positive about AI and looking to use them. Some of you aren't too thrilled with the tools and might avoid using them. I think that not learning to harness the power of an AI is a mistake. This technology is going to change the world in many ways and you need to learn how it can help you.
You also should learn where the sharp edges are as there are some very, very sharp edges.
Read the rest of Choosing an AI in Sept 2025
I get a regular email from Quora, with questions that I might be able to answer or might be of interest to me. There are some database questions, some Tesla ones, and others, but recently I caught an interesting one that asked: why are most people broke? To be fair, there are a lot of questions that appear to be trolling, not people seeking answers, but this one caught my eye. This isn't very data-related, but it is a bit of advice from someone who is getting older.
As I look towards retirement in the next decade or two, part of my planning is ensuring that finances are in order. Once I stop working or slow down, the economics of my family change, and we need to be thinking forward to ensure that we have enough funds for our daily living. This can vary in different parts of the world, but I'm surprised at how many people don't really think about covering their daily expenses without a salary.
Read the rest of Don't Forget About Financial Skills
As I get older, I find that time is the most valuable commodity I have. It's the one thing that I need more of, but I can't get it. Even if I find more efficient ways of doing things, it seems there's an endless list of things at work and on the ranch that need to be done.
It seems to be the same for most of my friends in other jobs, whether in medicine, law, or any other position; they're often overloaded with more work than they can get done in a week. Arguably, it's not all important work, as sometimes we might tackle a task, only to have our boss throw the work away or delay the project. That's annoying, but I also understand priorities change.
Read the rest of Getting More Time from AI
I studied economics in university, which isn't that close to database work, though I did have to work through linear regression problems by hand. I always enjoyed mathematics, so this wasn't a hardship. Until I purchased a PC that was capable of letting me do graphs and calculations in PASCAL and BASIC. Then I realized that my enjoyment wasn't that efficient or useful, and a computer could help me get things done way more efficiently.
Many of us work on systems that process tremendous amounts of data, something our organizations couldn't complete without computer hardware, efficiently or not. We just wouldn't be able to get the work done by hand. That's the main reason why downtime is such a problem in the modern world; we can't fall back to manual systems in many cases.
Read the rest of All the Costs of Downtime
One of the most prolific and popular authors at Simple Talk has been Phil Factor. He wrote many pieces on all aspects of database work and has probably written more articles on the Redgate Product Learning site than anyone else. He has entertained, informed, and inspired many database professionals in his many years as an author.
Phil, aka Andrew, passed away recently. This was a shock to many of us and a sad day.
Tony Davis introduced me to Phil, whom I always thought of as Andrew, many years ago when I first traveled to Redgate. Tony published a tribute to Andrew on Simple Talk and has many more fond memories of Andrew. If you ever get the chance to meet Tony, ask him for a few.
Read the rest of Remembering Phil Factor
Many of you reading this likely have an Availability Group (AG) set up on at least one database in your organization. Maybe not most, but many of you as this has proven to be a technology that many people like for HA/DR, upgrades, and probably other uses. As the technology has evolved from it's SQL Server 2012 debut, it has improved in many ways. This might be one of the few features that has received regular attention from the developers in Redmond across multiple versions.
That's not to imply this is a foolproof or bug-free feature. Numerous people have had issues with the various types of AGs. From setup to performance to scale, I've seen many people post questions and search for answers on how to get their system running smoothly and reduce any late-night calls.
Read the rest of Pushing the Limits of AGs
Take a moment and think about how much work you do each day. If you write code, how much time do you actually spend writing code around meetings, admin work, and other tasks. If you manage systems and develop scripts, how often do you get to work on a script, change it, upgrade it, or add a new feature? I bet it's a minority of time for most of you. Studies show that many developers spend only an hour or two writing code each day. I would guess it's lower for sysadmins or DBAs who incorporate coding into their jobs.
Now think about how much you enjoy each thing you do. I assume meetings aren't the most exciting thing you work on, nor is updating a document for others to read. When I wrote code as the main part of my job, solving the problem, trying different algorithms, experimenting with enhancements, those were the fun times.
Read the rest of AI Steals Joy
There has been a lot of news about air traffic problems in the US in 2025. I haven't had any delays due to this, though I've gotten a few messages in my travels that I might want to reschedule. There was an article that some of the technology still used in various facilities is old and needs upgrading. Old as in Windows 95 and floppy disks.
That's old, but obviously it still works. Even with the various accounts of problems, almost every day thousands of flights are managed successfully by the people who run these systems. They're not alone, as the article also points out that some other transit systems make do with technology that most of us would never think of using for any system.
Read the rest of The Security of Old Tech
I remember working at a large organization with a team of other IT Operations staffers. We rotated this one job every month amongst a few people, each taking turns, where we'd lose a day to update all the privileged passwords for our servers. This was before Managed Service Accounts and the cloud, when we were required to change these every 30 days and then store the new ones in an encrypted store.
What struck me when I got stuck with this wasn't the requirement to change every 30 days; that seemed normal. The thing that bothered me was how manual this was. As a former developer, I wrote some scripts to automated this, pre-PowerShell, and make the task easier on my fellow sys admins. I had scripts to generate a password, change it in AD, then print the pwd to be copied into our secure storage (no API there). This ran in a loop so I didn't lose a whole day to changing password.
Read the rest of Password Guidance
I was working with a customer recently that is trying to improve their processes. This was a large company, over 100,000 employees, though most of them aren't in the technology area. However, across many divisions and groups, there are a lot of developers and operations personnel who have tended to work in silos, managing their own applications and systems in disparate ways.
In other words, doing software development the way most companies do it.
Read the rest of Requiring Technical Debt Payments
I saw a post from Brent that Microsoft had changed the default memory guidance. At first glance I read this as they'd changed the default values, which would be interesting. However, this is a guideline, set to 75%. I also saw a few thoughts from Randolph West on LinkedIn, and quite a few comments. The comments were interesting in a few ways.
It is easy to look at 75% and say that won't work for this server that's on my mind right now because I keep getting woken up. That might be true. However, the 75% number isn't a hard requirement. It's a guideline, a recommendation to ensure you have enough memory for the OS, but you're trying to use most for SQL Server. Feel free to adjust it if you feel the need.
Read the rest of Guidelines and Requirements
One of the major concerns for using GenAI tools is who is reading the data you submit as a prompt, and will this data be used in future training of the model? In other words, could someone using a future model access the data I put in a GenAI chat?
It's a valid concern, and not just because of the vendors. There is a lawsuit over the use of data by OpenAI, and a court has ordered all chats to be retained, including deleted ones. Since this is a lawsuit, there is always a chance that some of the data retained gets entered into a court document or even that it might be read aloud in court and captured in a transcript.
Read the rest of Concerns over AI Chat Privacy
I remember the court case years ago when the US government wanted to access data in Azure that was physically stored in Ireland. I wrote lightly about this and linked to the article back in 2020. This has typically been more of a concern for the EU (and other countries) than the US, but I'm sure there are organizations in the US that use the cloud and don't want their data accessed by other countries' governments.
Recently, a Microsoft executive was asked about this in the French Senate. The Microsoft response was that they (Microsoft) cannot guarantee data sovereignty for French customers. If the US government served a warrant under the Cloud Act, a US corporation would have to turn over the data.
Read the rest of Data Sovereignty in the Cloud
Ghostworking is pretending to be busy at work. According to this article, it's on the rise with a number of people looking for other remote opportunities at other organizations. Some employees might be just wasting time at the office, though I suspect there is a minority that are actually working at another (second) job. Maybe they are growing a side hustle or maybe they've even accepted another job, and they're spending part of there day on those tasks.
This might be part of the reason that many managers want to get people back into the office. I think that's flawed as a) most managers aren't great at their jobs and b) people still ghost work in the office. I've had numerous "clipboard carriers" alongside me in my career who spend an inordinate amount of time at the water cooler, coffee machine, or walking from place to place and are happy to engage in hallway chats. I've seen plenty of managers also not know how to hold people accountable for their work (or lack thereof). Especially technical people who find many reasons why problems are hard to solve.
Read the rest of Ghostworkers
SQL Server 2025 is out in preview form. CTP 2.0 is available for you to test, and there are quite a few changes that have been added to the platform. Some of these are already in the cloud, but this is your chance to test them in your environment.
This is version 17.x, though I don't know we've really had 17 versions to work with. In any case, there have been a lot of versions in my career. I've connected to and worked on all of these: 4.2, 6.0, 6.5, 7.0, 2000, 2005, 2008, 2008R2, 2012, 2014, 2016, 2017, 2019, 2022. No 2025 yet (other than install and a few basic queries), but that's coming.
Read the rest of Your Favorite Improvement
I mean, you can ask for a raise. It's always an option, but is now the time to ask for a raise in this economic climate, where businesses can be unsettled and there are a lot of tech workers looking for jobs. I even saw a piece that noted many computer science graduates are struggling to get hired at the top tech firms. While that's not where most people work, it seems that often other CIOs/CTOs look to the big tech firms for guidance, and they may choose to hire fewer new staffers as well.
This is on my mind at work after reading an article on asking for a raise that I added to the newsletter. It's a response in the advice column where a reader asks how to go about asking for a raise when they think they deserve one. It is unclear in which industry or level this person works, so I am not sure if I think the advice is good.
Read the rest of Can You Ask for a Raise?
At SQL Bits, I was chatting with Brent about a few things, including AI, which we think is changing the world. I've got my set of AI experiments going, and I do believe we will fundamentally alter work and how we use computers in the future.
Not sure if it's for the better or worse, but things are changing and will change more.
Read the rest of The Next Great Thing
For many years, most of the SQL Server installations I managed or deployed to were Standard Edition. Even in large companies that had agreements with Microsoft, since each instance had a cost (even at a discount), we were careful about where we installed Enterprise Edition.
These days with Microsoft wanting to charge everyone for every core, it makes sense to use Standard Edition wherever you can.
Read the rest of Who is Using Standard Edition?
Often, we work on projects that might span a significant amount of time. Weeks, months, even years can be devoted to working on one system, or the same system. We may find that our motivation rises and wanes at different times. We get excited and motivated to get things done, and also become discouraged or distracted when challenges arise. Not all of us, or not all the time, but it is hard to maintain a high level of productivity and motivation over time.
For managers, it can be a challenge to keep a team moving over time and focused on achieving goals. Life gets in the way, people take vacations, staff changes, and other things are all issues that can distract a team. Keeping a team focused and productive can be a challenge for many managers. Perhaps especially challenging when someone hasn't received enough training on how to manage and motivate others.
Read the rest of Carrots and Sticks
There was a time when I knew a lot about the various storage technologies available for a database. It was important when designing a server around the various requirements for size and performance balanced against the limitations of hardware.
The rapid growth of solid state storage and the adoption of storage area networks have changed the game for many of us. We no longer care or think about storage. It's just a service that we consume in our databases, and while we might demand more IOPS capacity, we often don't worry too much about how that's provided to our systems.
Read the rest of Storage Enhancements
This is my last day of work. Not forever, just for six weeks. I'm off on my sabbatical after today and won't be back until August 11. However, everything should run smoothly with Grant and Kellyn holding things down until I return. Have a little patience with them as this site can be a bit of a hectic whirlwind at times, and they still have other jobs to do.
It's been a wild first half of the year. After very little travel in Jan/Feb, the rest of the year has been a bunch of travel, including most of May and June being on the road. With coaching responsibilities for two teams from Jan-Apr, I am ready for a break. No big plans, but I am looking forward to being at home, playing some guitar, working on a few projects while trying to be very unwired for six weeks.
Read the rest of A Well Deserved Break
I ran across an article on the 7 types of tech debt that can cripple your business, which is a great title. It certainly is one that might scare a lot of CTOs/CIOs/tech management. I am sure that much of the IT management gets concerned on a regular basis with how quickly their staff can evolve their software to meet new business needs.
The first two items have to do with data, which is understandable. Data is the core of how many organizations operate and move forward, and if you don't have the ability to easily work with data in a flexible way, you can struggle. Many of us technical people know this, but I find many non-data-professional staffers don't get this and are often unwilling to work at improving the situation. They things to just be magically better without changing how they do their jobs.
Read the rest of The Technical Debt Anchor
Each time I compile and curate the Database Weekly newsletter, I find lots of Fabric content from the various sources I watch to compose the newsletter. Since I primarily deal with the Microsoft Data Platform stack, this makes sense. Most of the things I am interested in are related to Microsoft, and as a result, I tend to use sources that also use SQL Server, Power BI, Fabric, and related technologies. I do look for other related data items, but I am heavily MSSQL focused.
Recently, I stumbled on a piece that contains Fabric Alternatives in AWS, GCP, and OCI. It covers some of the options on these cloud platforms at a very high level. A product name and short description, but it shows there are other choices. I found it interesting that Databricks is mentioned, but not Snowflake. I'm not sure why that is, as Databricks is on Azure (and other platforms) as is Snowflake, but perhaps the author doesn't consider Snowflake a peer? That seems strange.
Read the rest of The Data Warehousing Choice
Part of my Redgate work is with customers who need to monitor their database servers. With estates growing quickly, both in scale and types of database platforms used, keeping an eye on everything can be challenging. Add in the lack of staff growing as quickly are the number of servers, and I find many companies seeking out monitoring tools to better help them manage the entire estate..
When someone evaluates a tool, one of the first questions from many people is about load. They are concerned about the load a tool puts on the system, which is always some amount. Most tools say they use less than 2% of total resources, some might hedge at 5%. Hopefully, there's no more impact than 5%, though that might seem to high, especially if you have a busy database server already.
Read the rest of Multiple Monitoring Tools
Are you looking forward to SQL Server 2025? Or perhaps you think this is just another release, or perhaps you are not looking for new features or capabilities in your environment. Maybe you don't care about new things, but are looking for enhancements to features introduced in 2017/2019/2022. There is certainly no shortage of things that can be improved from previous versions (cough graph *cough).
I ran across an article on the five things that one person is looking forward to in SQL Server 2025. It's a good list, and the things included make me consider an upgrade. Certainly, any improvements in the performance area, especially with all the investments made in Intelligent Query Processing over the last few versions, are worth evaluating. They might help your workload, or they might not, but if they do, then upgrade.
Read the rest of SQL Server 2025 Excitement
I had to make a few changes to a SQL Saturday event recently. The repo is public, and some of the organizers submit PRs for their changes, and others send me an email/message/text/etc. for a change. In this case, an organizer just asked for a couple of image updates to their site. I opened VS Code, created a branch, added a URL for the images, and submitted my own PR. After the build, I deployed it.
And it didn't work.
Read the rest of Patching the Patch
When talking about DevOps, the goal is to produce better software over time. Both better quality as well as a smoother process of getting bits to your clients. There are a number of metrics typically used to measure how well a software team is performing, and one of the things is Change fail percentage. This is the percentage of deployments that causes a failure in production, which means a hotfix or rollback is needed. Essentially we need to fail forward or roll back to get things working.
For most people, a failed deployment means downtime. I've caused a service to be down (or a page or an app) because of a code change I made. This includes the database, as a schema change could cause the application to fail. Maybe we've renamed something (always a bad idea) and the app hasn't updated. Maybe we added a new column to a table and some other code has an insert statement without a column list that won't run. There are any number of database changes that might require a hotfix or rollback and could be considered a failure.
Read the rest of What is a Failed Deployment?
Many of us are faced with choices and decisions constantly in our jobs. How do we approach a problem? What should we do as a team to get the work done? How do we code or manage or test or do something else with a database?
Maybe more importantly, how long do we spend deciding?
Read the rest of Shorten the Debate
At an event recently, I had a chat with someone after one of my sessions. I had been speaking on DevOps and ways to better structure your team and build software. After the session, one person asked me if I'd read The Mythical Man Month and if I felt we'd gotten a lot better at building software since that book was published.
I do think we have gotten better, way better, in fact. I caught another review of the book a while back from the Pragmatic Engineer. That view looked at what's changed in 50 years since the first edition, as well as contrasting the world today. You have to subscribe to read that one, but I'll give you a few thoughts from me on the book itself and the review.
Read the rest of Reflecting on the Mythical Man Month
I've been reading an interesting book that looks at some of the ways that we can better build software in enterprises. One of the side notes in the book is that the tech companies have the funding and the ability to disrupt many other types of businesses, not just technology. Google, Amazon, Microsoft, and others have delved into other types of industries, potentially pushing others out.
We see Amazon becoming as much a shipping and logistics company as they are a retailer. There was a unionization vote, which passed in 2022. Recently, another one failed. Amazon continues to fight these efforts, trying to prevent workers from collectively negotiating the terms of their employment.
Read the rest of IT Unionization
At a recent event, I had a student ask about how to get started as a data professional. What types of things should they do? What platform should they work on or learn? Where should they focus time? What tools are available?
Those are all good questions and many of you likely have your own advice. I'll give a few things to think about today, which are good for anyone that might want to get into the data field. I think this is still one of the better technical careers. I've suggested this to my kids, though only one of them went into a technical area. One is a therapist for autistic children and one helps run the ranch while pursuing a graduate degree in literature.
Read the rest of Helping Students with a Data Professional Career
I saw an article on AI usage that is based on an upcoming book that suggests redesigning the world around new tech, not adding it to existing things. The first example is how electricity was introduced to existing factories, but it only provided some incremental gains until new factories were redesigned around electric motors. There's also an example given about reworking hotels to remove the front desk since that feature isn't needed. Instead, people could walk in, and an employee with a tablet could find them to check them in.
I'm all for rethinking and redesigning processes. I do think we have a huge glut of software in many organizations that exists because processes have evolved across time, but not everything and we keep our old software. When we have a new need or want a new capability, we add new software (or add features), but we don't necessarily throw out all the old software, processes, or habits. That wouldn't be practical, often because when we implement something new, it might not meet all our needs. Or at least we don't know it meets our needs at first.
Read the rest of Changing the Paradigm of Work
As a part of my job, I often work with customers on how they can get database code into a version control system. That's Git for the most part today, which is the most popular system in the world. I'm comfortable using Git for many basic tasks, but I am not an expert by any means. I've used version control for years, and quite a few systems, and I like Git as a way of managing code.
I have been surprised how many people aren't comfortable with version control or Git. Many don't have the habit, but are amenable to it. What I'm amazed by in 2025 is how many people don't use it, given that so many tools we use to work with databases, and even other systems, will store items in Git. This isn't just for development code, but also for infrastructure code. Lots of data tools and servers can store data in Git and use it to deploy changes to all kinds of systems. I'd have expected more people to know Git.
Read the rest of Does Version Control Scare You
It likely isn't a surprise to many of you that executives like AI. A survey shows that 74% of executives surveyed have greater confidence in AI-generated insights than advice from colleagues or friends. At the board level, even more (85%) favor AI-driven advice.
That's amazing to me, and while I might think this is a bit too much trust being placed in these GenAI LLMs, perhaps it's also partially because they work with too many people who aren't great at their jobs. Plenty of people skim through data or focus on certain things and might miss the details. While an AI can read and summarize a lot, it might not have the context we expect. I tend to be a bit skeptical of AI summaries, often because they don't necessarily weigh the different parts of an article the same way that I do.
Read the rest of The AI View From Above
I read a piece recently that got me thinking that data breaches might be inevitable. Disclosure: This was written by Redgate, for whom I work, titled "Data breaches May Be Inevitable—Compliance Failures Don’t Have to Be". It's based on our research with the State of Database Landscape survey as well as feedback and conversations with customers.
The thing that caught my eye was the first part: data breaches may be inevitable. Do you think that's true? Are we doomed to lose data in our organizations, not as a possibility, but something that will happen at some unknown time in the future? Those of you who have suffered breaches might agree with this, but for those of you who haven't had to deal with that situation, are you resigned to it happening at some point?
Read the rest of Are Data Breaches Inevitable?
Who among us has deleted a production database?
I'd hope it's very few of you that have done this in your career. I'm sure a few of you have deleted (or truncated or updated all rows for) a table in production. I've done that a few times, but fortunately, I've been able to recover the data quickly. I had this happen in SQL 6.5 and was grateful I could start a single-table restore before my phone rang.
Read the rest of Deleting a Database
The DORA organization is constantly researching how to better produce software at any organization. This is similar to work done by Carnegie Mellon Software Engineering Institute many years ago. Both groups are trying to determine what things help engineers work better and produce high-quality software.
On the DORA site, there is a database change management page, where they recommend approaches to managing database schemas. The main thing they talk about is treating all schema changes as migrations, which is something some people do. However, many teams also like a desired-state-configuration approach, where they just deploy all the changes from dev (or QA) to prod in a state-based flow. Both can work, but I do think as software matures (and becomes legacy), migrations are preferred. The article lists lots of frameworks in different languages. Flyway is among them, which is the product on which I work and sell at Redgate. If you haven't looked at a migrations framework, I'd recommend you do so. They do really give you a tremendous amount of control and flexibility. There are trade-offs, so I'm not blindly recommending this approach, but it's worth educating yourself on how migrations frameworks work.
Read the rest of Database DevOps Recommendations
I'm sure many of you have tried a GenAI LLM to do something. Maybe write some code, maybe get some sort of recommendation or suggestion, maybe to rewrite something or summarize text. I'm sure you have had some feelings about whether the tool made you more or less productive.
There was a trial conducted by the Australia Department of the Treasury on Microsoft's 365 Copilot, asking for volunteers to participate and use the tool in their daily work. They used it and then completed a survey, which are summarized in this piece. Only 218 people went through the trial, and the results are interesting.
Read the rest of How Helpful is a GenAI Copilot
Certifications can pay off, at least in some areas. There's a piece that talks about pay rising for some tech professionals when they have some credentials. Networking, architecture, and project management are mentioned in the report, as are database and data management.
I haven't seen the source report, and I'm not sure which certifications are getting higher pay rises, but I'd guess that newer data technologies, things related to cloud databases, and even anything related to AI is likely to get you a raise.
Read the rest of Rising Pay for Certifications
I thought this story about a programmer and a GenAI to be rather humorous. The individual was a game programmer and used the Cursor AI assistant to help them generate some code for a game. After a few hundred lines of code, the AI delivered this: "I cannot generate code for you, as that would be completing your work. The code appears to be handling skid mark fade effects in a racing game, but you should develop the logic yourself. This ensures you understand the system and can maintain it properly."
That response makes me actually smile to myself and chuckle out loud. I likely wouldn't feel the same way if I were asking for help with some code to handle a task like this, but it's kind of funny to have the GenAI stop and say this. However, it's akin to the RTFM response plenty of humans have given others when they were asked a question. This is pointed out in the article as the type of response plenty of people see on sites like Stack Overflow. Fortunately, I think we've avoided a lot of that response on SQL Server Central.
Read the rest of Learn to Code
When is the last time you interviewed for a new position? It could have been at a new company, or maybe you had an interview was for a different position inside your existing company. Perhaps you needed to talk to a manager internally for a new project. I've tried to treat all my one-on-one meetings or reviews as interviews since I'm usually trying to impress someone enough to get a raise or promotion.
Preparing for something you do rarely is hard. Most of us have interviews very infrequently, and we often aren't prepared to impress others. If your partner or a close friend is also a business person, perhaps they can help you get ready, but I've found that most people struggle to help others prepare as they don't know how themselves.
Read the rest of Interview Tips
I heard Brent Ozar recently talked a bit about the SQL Server platform and its future. He also mentioned that Fabric has distracted the data platform team and it isn't a great product. I tend to agree, and I see too many bugs, holes, and problems. However at the end of this short snippet, he talks about SQL Server with an interesting comment.
Is SQL Server feature complete?
Read the rest of Is SQL Server Feature Complete?
I get the Gates Notes email periodically and I always find it interesting to read. Like Bill Gates or not, he is a very smart individual and has thoughtful things to say. Even when I don't always agree with him, I enjoy hearing his view and have enjoyed seeing him deliver presentations. In fact, one of my career highlights was at SQL Saturday #175 - Fargo, held at the MS campus. Bill Gates was speaking to employees that day and we were allowed to watch the Q&A from the balcony. Later, I saw him start to leave and stop by a sign. He asked someone about SQL Saturday. When they explained the idea for free conferences, he said "that's cool."
One of the recent emails talked about the 50th anniversary of Microsoft, with the original source code available for a BASIC interpreter. It's an interesting read to me, since I learned BASIC first (and a little assembler) on an Apple II and a TRS-80. I didn't start a company, but I certainly appreciate the excitement of tackling a programming challenge back then when memory and disk were in short supply. Most of my early programming tasks had me worried about how much memory and disk I was using, trying desperately to minimize both.
Read the rest of 50 Years of Microsoft
At the end of last year, I ran into a friend I hadn't seen in a long time. We were chatting and this person mentioned that they were looking for a new job. They had been laid off and needed something. This is someone with a lot of experience and skill, so I wasn't worried for their career or future. At the time, they mentioned they had gotten an introduction and interest from Amazon, but they weren't interested in a position because of the return-to-the-office (RTO) mandate that Amazon was implementing.
I was recently chatting with another friend at a different company. This person manages a tech team, and was looking to hire another data engineer, but was told they could only hire in a certain city (City A) in the US. In this case, it was the city with their main office. They have offices in a few cities, and a large one in City B, but the organization has been thinking of their own RTO plans and has limited hiring. My friend is now wondering if they need to consider moving to City A (not likely) or find a new job. They don't want to have to go to the office every day in City B.
Read the rest of The Return to the Office Debate
The revolution with GenAI has been quite the ride since 2023 and quite a few people have been concerned that their employment status might be in jeopardy. I can certainly understand that, especially in light of the tight budgets, widespread layoffs, and executive views on AI technologies.
There was an article recently talking about AI taking over some jobs with a few tips on how to stay employed. While tech workers weren't mentioned as being vulnerable, repetitive data-heavy jobs, such as data entry clerks, telemarketers, and cashiers were. That last one is interesting. Lots of companies have tried to use automated checkout stations, but this hasn't necessarily eliminated cashiers. Maybe there are fewer, but lots of companies in the US have rolled back some of these efforts as fraud, mistakes, and slower checkouts have been an issue.
Read the rest of Staying Employed
One of the challenges in software development is coordinating database and application changes when one depends on the other. I find many software development teams struggle with this, especially in today's environments when no one wants to take a system offline. While some companies can stage and manage deployments, many of us find our systems need to keep running 24x7 with minimal outages (if any).
Lots of you work in environments where your software is changing on a regular basis. Plenty of you will either be developing those changes, or managing the systems to which those changes are deployed. You likely will be coordinating with other people (in either case) to deploy a software artifact (C#, Java, Python, etc.) and a set of database changes in order for your clients to use whatever new functionality is being delivered.
Read the rest of Database First or Application First
I saw an article recently that a quarter of the Y Combinator startups have 95% of the code in their repos being AI generated. The article notes that if no other startups had any AI generated code (no idea the likelihood here, then about 24% of their code for startups is GenAI written.
24%
Is that high or low? If you think about all the code you've written in the last year, how much of it could be reasonably generated by AI? All the queries, schema changes, test code, dummy data insertions, refactoring to add a column to a table or result set. Could it be AI written?
Read the rest of How Much AI Code Would You Use?
Some of you reading this are database administrators (DBAs) who manage systems as their full-time job. Others of you might be developers, analytics people, or someone else who has another job, but you get stuck with managing the database somehow. I've seen a receptionist and a dental hygienist act in this role. We may call you the accidental DBAs, though that doesn't imply you are good or bad at managing databases. I got into this line of work as an accidental DBA who was also a developer.
No matter what your job title, my guess is that you aren't over-staffed at your organization. Likely you wish you had one (or more) more person to help keep up with the work. It seems that we never have enough time to get everything done in a week. And that's with a full staff. What do you do when someone is sick or goes on vacation? If you're like me, you get further behind and feel extra stress while your coworker is out of the office.
Read the rest of Part-Time DBAs
The use of feature flags in software development has become more and more prevalent over time, especially as teams move to DevOps-style development with frequent releases. I've often thought that using feature flags allows technical people to separate out the deployment of some feature or change from the release of that to users. There are a number of articles on this style of work (feature flag driven development, Why Use Feature Flags?) as well as a discussion at Reddit.
I am a big believer in feature flags helping with improving your software in many ways. These articles (and others) highlight the advantages that a software organization gains by using feature flags. Failed releases become less of an issue, as the specific change that doesn't work can be turned off. This can even work with databases. I can deploy a database change and at a later time have the code (or new table/column) start being used when a feature flag is set. If there is an issue, I can turn off the feature flag and stop using the code (or populating the schema). I can then clean things up, even saving data before I make a change.
Read the rest of Using Feature Flags
A domain is a set of possible values (among other definitions). I use this word a lot in my work, often with a problem domain (the thing you're trying to solve) or the domain of possible values (like the US States and Territories list). That last one is interesting, as this is often the set of data we stick in some reference or lookup table to use in a form on a screen.
There is a domain as part of the SQL specification, which I never knew about. I was reading an article from Joe Celko on the CREATE DOMAIN statement. This doesn't exist in SQL Server and T-SQL, but it has been a part of PostgreSQL for quite a few versions. The article talks about the definition of a domain from a few very experienced database design people.
Read the rest of A Domain for Data
The DORA organization is dedicated to helping others build software better and faster, at a higher quality, and in a way that is more efficient. They continue to compile and publish the Accelerate State of DevOps report every year, which is a fascinating read.
As a part of the report, they have identified four key metrics that identify high performing organizations in terms of software. These are divided into two areas: throughput and stability. Throughput measures are change lead time and deployment frequency. Stability measures are the change fail percentage and failed deployment recovery time.
Read the rest of Database DevOps Metrics
If you're a fan, no. You're chomping at the bit, waiting to get everyone using GenAI models in their work. If you're skeptical, then you might think it's never coming to take your job, be a personal assistant, help you with coding, etc. Pick the task it won't help you with.
A more nuanced view, which is similar to mine, is from Kendra Little, in which she says AI will eliminate DBA Jobs Faster Than You Think, I'm not quite as pro-AI as Kendra, mostly because I see so many companies that are slow to change, slow to adopt new tech, slow to adjust their thinking. They just soldier on and keep running their business, as they've been doing for decades. Even when you might make a case for change, or prove it's worthwhile, they just don't spend the time to change.
Read the rest of Is GenAI Coming Faster Than You Think?
Well, not really the end. I doubt anyone running SQL Server 2019 is going to stop (or upgrade) just because mainstream support ended. Actually, I wonder how many of you know that SQL Server 2019 passed out of mainstream support on Feb 28, 2025. I do think the 6 or 7 of you running Big Data Clusters likely knew this was the end of any support.
I saw a report in the Register on this, which includes a survey of which versions are still running. This is from an IT asset firm and matches Brent Ozar's Population report. 44% of you are running SQL Server 2019, which is the largest percentage. Since there's an additional 32% of you running versions older than 2019, I'm sure that upgrading isn't a priority.
Read the rest of The End of SQL Server 2019
It's no surprise that our systems are under attack by all sorts of criminals. Some organized, some opportunistic, some just aiming for vandalism. We need to protect our digital systems to prevent issues, and a part of better protection is reducing the number of places that are vulnerable. Those places include databases.
This article discusses the rising costs of data breaches and the increased frequency of attacks. It also examines the increasing number of regulations that are demanding proof of stricter security measures. It can be hard enough to defend production systems, let alone protecting dev/test environments. I see an increasing number of organizations that limit access to production systems, even to the point that this impedes some of the daily work habits of technology professionals, but that is probably a good thing. Too many of us are too lax when it comes to security.
Read the rest of Lower Your Attack Surface Area
This is a strange time for many technology professionals and their employment. It seems many organizations have open positions and are struggling to fill those slots with qualified candidates (one look at this). At the same time, many other organizations have followed the large technology firms (FAANG, MSFT) and laid off large numbers of their staff in the last year.
At the same time, with the hype and rapid growth of GenAI systems, we have companies that are looking to machines to fill some of their labor needs. Salesforce noted that they did not expect to hire more software engineers in 2025 as their plan was to use AI tech to help them write enough code. They've seen productivity gains with AI tech, so they aren't adding more developers. They still need human developers, just not more of them.
Read the rest of The Job Outlook for Database Professionals
Recently there was some online complaints about social security numbers (SSNs) in the US being duplicated and re-used by individuals. This is really political gamesmanship, so ignore the political part. Just know that social security numbers appear to be one of the contenders used in many data models.
I found a good piece about how SSNs aren't unique, and have a mess of problems. Despite this, many people seem to want to use SSNs as a primary or alternate key in their database systems. They also aren't well secured in many systems, even though we should consider this sensitive PII data.
Read the rest of A Poor Data Model
I've been working with databases for a long time and there are no shortage of things I've seen other people do that I don't like. Sometimes I shake my head a little. Sometimes I might groan inwardly (hopefully not aloud), and sometimes I might make an effort to convince someone else to do something differently.
Sometimes I'm really annoyed (or angry) and don't even know what to do.
Read the rest of Your Biggest Data Model Complaints
Erin Stellato, a program manager at Microsoft, asked a very interesting question on LinkedIn: "why do you not want a Copilot in SSMS?"
That got me to stop and think a minute. Why don't we want to use an AI to help us? It's a good question, given the hype and (maybe) potential of the technology. While it might not help you now, or with your specific thing, it might help others, so are there good reasons not to use AI technologies, like the GenAI LLMs?
Read the rest of Why Not Use AI?
I caught an article on AI skepticism and there was a point in the article where trust was mentioned. Specifically the reasons that people distrust an AI or tech tool is that it makes a mistake, so they stop using it. A few examples of this were using a writing AI that made a grammar mistake or a GPS routing device that added a wrong detour. In those cases the humans stopped using the assistance of the algorithm because they felt it wasn't trustworthy.
What's fascinating to me is that I had this same conversation with a human the day before. Someone mentioned they were working with a group and they misstated something. After that, the group stopped listening to all this person's advice, thinking it was all suspect. Essentially one mistake overrides everything else.
Read the rest of Trust is a Funny Thing
I don't know how many of you will be disappointed or impacted by this, but Azure Data Studio (ADS) is being retired, as of 6 Feb, 2024. It will be supported for a little over a year, until 28 Feb, 2026. On one hand I'm not surprised, and on the other, I'm a little shocked by this.
I have written a number of articles on ADS, and shown how things work, as well as pointed out a number of things that don't work well in the product or its extensions. These pieces have gotten a number of reads, and people have commented on them, so I wonder if there are a lot of you that are upset by this. Is this going to change the way you work? I will say that it will lightly change my work, as I do use ADS to connect to PostgreSQL, but not so much for SQL Server.
Read the rest of The End of Azure Data Studio
I had never heard of data debt until I saw this article on the topic. In reading it, I couldn't help thinking that most everyone has data debt, it creates inefficiencies, and it's unlikely we'll get rid of it. And by the way, it's too late to get this under control. I somewhat dismissed the article when I saw this: "addressing data debt in its early stages is crucial to ensure that it does not become an overwhelming barrier to progress." I know it's a barrier, as I assume most of you also know, but it's also not stopping us. We keep building more apps, databases, and systems, and accruing more data debt. Somehow, most organizations keep running.
The description of debt might help here. How many of you have inconsistent data standards, where you might define a data element differently in different databases? Maybe you have duplicated data that is slow to update (think ETL/warehouses), maybe you have different ways of tracking a completed sale in different systems. Maybe you even store dates in different formats (int, string, or something weirder). How many of you lack some documentation on what the columns in your databases mean? Maybe I should ask the reverse, where the few of you who have complete data dictionaries can raise your hands.
Read the rest of Data Debt
If your job as a developer or DBA has been like mine, it's a constant stream of requests to change something, often without enough information and short deadlines that create a bit of stress. There's always more work to be done, and while it might be a great job, you're often trying to finish something quickly enough to get to the next thing.
In this mode, how often do you think about creating (or modifying) the thing you're working on for today vs maintaining it for tomorrow. In other words, do you consider how easily your work can be understood, is documented, is designed to allow for flexibility, and can be enhanced without many (any?) side effects, or anything else.
Read the rest of Creating vs. Maintaining
The short answer is of course, most of us can learn and improve our skills to become better developers, engineers, DBAs, etc. While we might not be able to become the 10x engineer that many aspire to be, we can certainly become a better employee inside of an organization.
There's a piece on becoming a more effective engineer, which is actually titled know how your org works. It a piece from an engineer that started with a tweet: The text was:You can either complain and pontificate on Twitter on how the tech industry should ideally work, or you can learn how your org really works and what’s rewarded, and optimize for that. Or quit and find another job. This might sound cynical - but it’s what it is.
That sounds a little harsh, but the reality of how your org works or is structured or interacts is a reality. We all have hindsight to look back and wish someone (including us) had written code better. We might be sure if we could change one thing, or add/remove someone else, or make some other change, then things would be better. We might feel that there is a simple solution. Those things might be true, but they aren't the reality of the situation.
Read the rest of Can You Become a More Productive Engineer?
There are many, many choices for cloud database services these days. I would hope everyone is aware of the various IaaS options in public clouds with EC2, Azure VMs, GCP Compute Engine, and others. These are often the easiest way to move your workload, but you've really just moved a VM from one place to another (likely more expensive) place.
For managed databases, there are lots of choices, but you might not be aware of your options. I ran across an article that discusses the various flavors of managed databases in the big three public clouds for SQL Server. In the piece, there is a section that talks about when a managed database makes sense. I like that it discloses the development on a managed service is expensive.
Read the rest of The Managed Cloud Database Options
Starbucks has too many order combinations. This article about the challenges for their new CEO notes there are over 170,000 combinations of things customers can order. As someone who has visited a store often and stood behind someone placing an order, that sounds like an accurate number. Sometimes I hear people order things to drink that I had no idea were even possible.
When there are too many choices, sometimes there can be a delay as the person tries to process the information and make a decision. Some people can't make a decision. Others enjoy the variety and keep making different decisions each time. That's fine if you're visiting a restaurant, but it's less useful if you're building software.
Read the rest of Simple(r) Coffee
I caught this post about a career presentation from Kendra Little and it resonated a bit with me. The summary of the post (it's a bit long) is that there is valuable work that supports and benefits a team of people, called glue work. However, glue work isn't necessarily technical and it isn't necessarily recognized as valuable by management. In fact, it might unintentionally lead you from a career in engineering to one in project management.
I haven't been someone who has been marginalized in positions, who has been taken for granted or expected to do certain tasks. I think some people get taken advantage of, and this is especially true for women, who both volunteer more and are volunteered by others more for these tasks. While I have been quick to document decisions or actions, and sometimes help out with project work, I do that in addition to any technical work. I've felt that as a developer, my job is to produce code. If I can help with other things, then I'm a Developer+, but I ultimately know that code my clients can use is what matters.
Read the rest of Adding the Right Value
For the last few years, we've seen no shortage of cloud migration stories and felt pressure from management who wanted to migrate our systems to the cloud. It seems that almost everyone I speak to has a story of having to move a system out of their owned or leased data center into a public cloud from some vendor. A lot of this is the movement of VMs from one place to another, which has me scratching my head. If we're just running VMs, surely we can do this cheaper in our own data center.
Perhaps, though there are a lot of costs to setting up or running a data center, and it's not easy getting a system in place that allows a bit of self-service for our customers. Especially while ensuring that images used are properly patched and secured, while ensuring lots of easy connectivity to storage that can be reconfigured easily. It might not be worth the effort for a few dozen VMs, but if you have hundreds of systems, maybe it is.
Read the rest of The Rise of Data Centers Centers
I was chatting with a friend recently about routines and some of the helpful or silly things we do. I mentioned that when I played adult baseball, we often had Sunday morning games and a routine of mine was to drive to town, stop at a 7-11, and get a large cup of coffee and an apple fritter. It was a comforting habit that I still have today, often stopping when I have a morning flight to do the same thing in the way to the airport.
Only in Denver though, not when I'm flying out of other cities and returning home.
Read the rest of Comforting Habits
I went to San Francisco for Small Data SF, a conference sponsored by Mother Duck. The premise of the event was that smaller sets of data are both very useful and prevalent. The manifesto speaks to me, as I am a big fan of smaller sets of data for sure. I also think that most of the time we can use less data than we think we need, especially when it's recent data. That often is more relevant and we end up with contorted queries that try to weight new or old data differently to reflect this. Maybe the best line for me is this one:
Bigger data has an opportunity cost: Time.
Read the rest of Big Data or Small Data
When I was younger, I had a variety of jobs, but in most of the positions I had to work hard for stretches. Really hard, as in more than 8 hours a day or 40 hours a week. Often as I was starting a new position, it took some time for me to develop some understanding, some skill, and some muscle memory. In some jobs, especially in restaurants, I also had to build the physical skills to be on my feet for many hours.
In technology, I've often found myself unsure of how to approach a new position, aware I had knowledge gaps about how things worked, and often, I was naïve or ignorant of some piece of technology my employer used. Even at jobs where I started as a developer or DBA on a known platform (ASP or SQL Server), I sometimes encountered some aspects of the technology that I hadn't used in the past (like clustering).
Read the rest of Learning to Grind
I was asked this question recently: is it more likely that AI will replace humans or assist them in their work?
It's a good question. If you think about the way AI is being hyped in 2024, many people think AI is, or will soon be, replacing people and we need less of them in work. I guess the simplified view is that AI can do the jobs of many people, but I'm not sure the world is that simple. What I think is more likely is that AI becomes a lever that assists a few people in getting more work done and potentially replacing other, less knowledgable humans.
Read the rest of The AI/Human Spectrum
It seems there's quite a dichotomy in the technology workforce. On one hand I hear about the Great Resignation where many employees are leaving their jobs because of RTO (return to office) mandates or some other dissatisfaction with their job. On the other, I've seen quite a few people who were laid off and are struggling to find new positions. On the third hand, there is no shortage of companies who report they are struggling to find and hire talented people for some positions.
I don't quite know what to think, but I do know that employment seems to be harder to come by. If you are looking for a job, I believe you need your own sustainable learning plan. If your organization can't find qualified talented staff, then the organization definitely needs a continuous learning plan.
Read the rest of Continuous Learning
I was listening to someone at Microsoft talk about their product recently. I can't disclose which one it was, but lots of people use this product and are impacted by changes. The particular thing that caught my attention was that the presenter noted there was a breaking change in the new version for some people. This wasn't a huge change or one that would affect a lot of people, but it was a breaking change for a few.
Another attendee asked this question: I would tend to do xxx instead of what you showed, so why would you do this?
Read the rest of Why Would You Do That?
I was watching a video called Microservices are Technical Debt. In it, the person being interviewed said that a lot of people really have a distributed monolith. That caught my eye since I've worked with a number of customers who are trying to adopt microservice architectures for their applications. I think this is less a performance/scaling choice than a reworking of their software development teams, and I'm not sure they will end up with a better system.
What is a distributed monolith? I am not an expert, but this appears to be a place where all the services still depend on each other. For example, I might have a service getting user profile info that an app calls, with another service getting previous orders, and a third service that returns inventory. In a monolith, if any of these are down, the others don't work. In a distributed monolith, these might be built independently, but perhaps the core app/web page still requires all these to be working to show the user something.
Read the rest of Distributed Monoliths
There are lots of software development methodologies. This page lists a few, among them waterfall, agile, iterative, rapid, and more. What's been interesting to me is that the process of deciding what to code and then whether it works doesn't change much between different ways of building software.
Instead, the cycle time between when we ask a client what to do and when we deliver it changes. The more agile/lean we are, the lower the cycle time. The more waterfall-ish, the larger the cycle time. I guess that analysis and breakdown of problems into work also changes, as the scope in modern DevOps styles of development is smaller (more contained) than in waterfall.
Read the rest of Reducing the Cycle Time
I've been very pleased with the direction of SSMS the last few years. As it's been separated from SQL Server releases and gets updated more often, I think the changes from v17 though v20 have been improvements. There are still issues, but it's been better. Now we finally have SSMS moving to a modern shell with the v21 preview and I'm excited to see how this changes the future of our tooling.
However, the PM for SSMS, Erin Stellato, posted a note on LinkedIn recently asking why people don't read documentation. She also asks what you want to see in 21, so respond if you think there are holes in the SSMS docs. I think this post came about because of many responses that came from people who clearly hadn't read some documentation.
Read the rest of Doing a Little Research
Thanksgiving is tomorrow in the US and it is supposed to be a holiday when we give thanks for our blessings in life. My wife usually has everyone in our family tell what they are thankful for this year. I also see many people posting things they are grateful for during the month of December.
Last month I was lucky enough to have dinner with Bob Ward and we were talking about some of the things we'd seen in travels, often some stressful times for ourselves or others. We've seen many people get upset or angry or have some other reaction. Both of us have some empathy for others, recognizing that we don't really know their history or experiences, and it doesn't make sense for us to get upset. If we knew those things, we might better understand the reaction that someone displays to a situation.
Read the rest of Have Grace
I was a bit of a math nerd in high school and college. Some of you might have been as well, but I took advanced math all through high school, culminating with AP Calculus as a senior with 11 other kids (of about 320). In college, I started with Calculus III freshman year and went on to take 7 more semesters of various high level maths. One of those classes included analyzing data with linear regression, which we did with hand calculators and formulae.
At SQL Saturday Pittsburgh 2024, I watched a talk from Jeff Moden on linear regression. It was a trip down memory lane, with Jeff explaining how the process worked, the flaws, and how this technique could be used to do some predictions on data stored in SQL Server. It was a great session on the topic, but I liked that Jeff showed how you can use SQL Server to do various math calculations that might be useful to analyze data. I see applications sometimes programming various formulas, but I don't often see people doing this in database queries.
Read the rest of Computer Algebra
In most of the organizations I've worked for or consulted with, patching was always a challenge. Patching hasn't usually been given a priority and is often skipped when operations staff is busy. This has resulted in lots of un-patched, or slowly patched systems. I assume this is one reason Microsoft continues to release RTM-GDR patches because some people won't patch at all unless there are critical fixes.
I also know that much of IT management sees patching systems like patching parking lots. Needs to be done, but tomorrow, after we do other important work today.
Read the rest of How Do You Patch 100 Database Servers?
Microsoft constantly releases new features and products in the data platform space. Many of us have seen the SQL Server product grow in new ways, some of which are very useful to us. As an example the changes from log shipping to clustering to Availability Groups has improved our HA/DR options as well as the capabilities available to us in different situations.
With that in mind, I saw someone recently that wanted to deploy SQL Server on Kubernetes, which is something that could be a very interesting way of managing your different systems. However, this individual wanted to know when Microsoft would release their own supported solution with a Microsoft operator to manage the instance. There is guidance from Microsoft, but no official operator.
Read the rest of Do You Want a Microsoft Solution?
Recently I was watching a presentation on how to scale performance in your SQL Server environment and one of the suggestions was setting up Availability Groups (AGs) and having read-intent connections that would query the secondary and not the primary. It's not a bad idea, and the SQL Native Client (and other drivers) support this and make it easy to implement.
The pattern of using multiple connections in an application, one for reads and one for writes, has been suggested often. However, in practice, I've rarely seen this work. Apparently having a connection variable, named dbConn, for writes and a second one, named dbConnReadOnly, for reads is too complex for most developers or teams.
Read the rest of Separate Reads
I assume most of you work with others in a team. Even if you are the data specialist and others work on different technologies, you still have a team. How long has your current team been together in this form? Have you had a stable team that might have grown, but the rest of the individuals and roles/responsibilities stay the same? Or has your team changed makeup, roles, responsibilities, or something else?
I don't see a lot of organizations that change their team structures often. There may be people who come and go from a team, but the core structure remains the same. Even when your company might reorganize a bit, often it's teams that shuffle between managers, but mostly remain the same. There certainly are exceptions, and some large orgs (Microsoft, Amazon, etc.) regularly shuffle lots of people around, but I'm not sure the teams change their makeup or their mandate much.
Read the rest of Time to Change Your Team
Recently I had a friend traveling who is not very tech savvy. This person has traveled before and has a routine, but in this case, they were struggling to get an airline's mobile app to work. They also struggled with the website, and just before the trip, they were thinking to cancel because they didn't have a ticket in their hand before driving to the airport.
This turned out to be a login issue, and between friends and the airline's customer service, they were able to print out a ticket at home and take it in hand to the airport.
Read the rest of Technology Fears
Today is the first day at the PASS Data Community Summit and I'm in Seattle where I'll get to open the conference and introduce the Microsoft keynote. I'm sure the keynote will be full of announcements on something, but what?
I'm writing this a little over a week before the event, and I have no idea what Microsoft will do. Actually, by the time you read this I may have some ideas as there was a practice session yesterday, but I can't tell you anything. NDA, and really, by the time I got something organized, the keynote will be done.
Read the rest of What's New for the Microsoft Data Platform
For a long time I've felt that my recommendation for people wanting to enter technology wasn't to go to college and get a degree, but rather start to learn on your own and get an entry level job (help desk, tech support, etc.) and start to work in the industry. That's a good way to both experiment and understand what you're considering undertaking as a career, as well as limiting your investment. It's also nice to get paid to learn something.
College is great, but it's also expensive. I find that for many people, it can be hard to get a good ROI from college these days. The fast rising cost, not to mention the uncertain opportunities after college lead me not to recommend pursuing a CS degree, or really any degree, as a default view. There are exceptions, but for many people, I'd prefer to work and try to better understand where they should invest in education.
Read the rest of I Need a CS Degree. I Don't Need a CS Degree
Your management gets a great demo from a cloud vendor and decides that the organization needs to implement the new service/application/etc. quickly. Your team tries to comply, furiously learning and experimenting with integrations, software changes, infrastructure configuration, and more. Things get deployed are working. Clients and management are happy with the new capabilities and you breathe a sigh of relief.
After a bit of time there's a security issue and all of a sudden there's blame pouring down on everyone. The vendor takes a hit because it's a public security problem, but the reality might be that your organization didn't completely understand how to configure strong security. The public doesn't blame your organization, but internally your team don't know how to make changes to ensure future security.
Read the rest of The Cloud Security Problem
There's an article at Forbes about the Five Things Business Leaders Should Know about Databases. Disclosure, it's by my boss, but I think it's still a good read. These are points we've learned from research and work with customers and prospects at Redgate Software. These points come from you, as well as from executives with whom we work, but there are so many people in organizations who don't think about the complexity of data, so it's a good one to pass along.
The five things are (if you don't want to read): data is growing, getting more complex, there are multiple database platforms in most estates, teams struggle (duh), and data is a business issue. Most of us know about the fourth one, often because we may feel overloaded with work. We might also feel a lot of stress in trying to keep up with not only the workload but also trying to learn more to support the ever-growing variety of systems it seems our employer wants to put into production. I regularly talk with customers whose developers keep wanting to try out a new, shiny database platform in the cloud (or add new features from their existing platforms).
Read the rest of Databases for Executives
For much of my career as a younger person, I was mostly concerned with salary at a job, along with the opportunities for my career. I really wanted to know how much money would hit my bank account and cared most about that. I also wanted to know if I would learn something or get a better title or work with a technology that might help me in the future. That drove me through quite a few jobs in my 20s and 30s, leaving some for more money and more opportunity.
As I got a family, I became more concerned about healthcare since that industry is a mess in the US. Often when I looked at a job, I perused other benefits but didn't give them much weight, mostly concerned with salary and the cost of medical insurance. I also somewhat cared about who I worked with (the team), but that was more for helping me choose between different jobs. It wasn't something I thought of as a reward, though I should have.
Read the rest of Knowing Your Total Reward
I own a Tesla, which is essentially a computer on wheels. Much of the way the car works is driven by software, which I love. New features have appeared and minor fixes come through in the same way that they do for apps on my mobile device. It can be annoying to wait for an update to install, which has happened when my wife or I start the update remotely and don't realize the other is planning on driving. Fortunately, I can set these to run overnight from my phone and they mostly disappear into the background.
I don't worry about Tesla failing, at least, that hasn't been on my mind, but I ran into this article about a company in China that is failing. The WM Motor Company filed for bankruptcy, and perhaps coincidently, their app stopped working. Owners couldn't manage basic functions. The company put the server back up, but that brings up a bit of a concern for software that depends on external connections.
Read the rest of When Companies Fail
My wife and I have been thinking about some new audio equipment. We've been a little unhappy with our Bose soundbar because of the software flakiness and sporadic network connectivity issues. In looking around, I saw a Sonos product, but after reading a bit about the company's recent history, I decided to look elsewhere.
Sidebar: if any of you have recommendations that aren't high-end $$$$ audio, let me know.
Read the rest of Tech Debt Perils
At the Small Data conference recently, one of the talks looked at hardware advances. It was interesting to see a data perspective on hardware changes, as many of us only worry about the results of hardware: can I get my data quickly? In or out, most of us are more often worried about performance than specs. However, today I thought it might be fun to look at a few changes and numbers to get an idea of how our hardware has changed, in the march towards dealing with more and more data. Big data anyone?
In thinking about disks, I saw a chart that looked at the changes from HDD (hard disk drives) to SDD (solid state drives) to NVMe (Nonvolatile Memory Express). These show read speeds going through the list from 80MB/S to 200MB/s to 5000+MB/s. That's a dramatic change, and not one only in high-end arrays. There are off-the-shelf drives you can put in a desktop that read this fast. If you think about some of the early IBM drives, which read at 8800b/s. Growth in disk speed, inside the timeline of our careers, has grown by a few orders of magnitude in read speed.
Read the rest of The Vast Expansions of Hardware
These days algorithms rule much of the world. From how supply chains are managed to how vehicles run their engines to the media that many of us watch on the various streaming services. I assume that most of you know that algorithms drive what you see on social media, on YouTube, and even the search results you get, and what you see might be different than what I see. There is a constant search for a perfect, or at least, very targeted way of getting you what you want.
Or at least what the algorithm thinks you want. However, is that the best way for algorithms to be designed? It is for the companies that want to profit from your attention, but is this intense personalization better for us?
Read the rest of The Modern Algorithm of Chance
I'm hosting a webinar tomorrow with this same title: The Role of Databases in the Era of AI. Click the link to register and you'll get some other perspectives from Microsoft and Rie Merritt.
However, I think this is an interesting topic and decided to try and synthesize some thoughts into an editorial today, partially to prep for tomorrow and partly because I'm fascinated by AI and how it will be used in the future.
Read the rest of The Role of Databases in the Era of AI
I posted a note on Twitter/X with this quote: “The content updates had not previously been treated as code because they were strictly configuration information.” This is from testimony given by Crowdstrike to a US Congressional committee in trying to explain how they grounded much of the airline industry a few months ago. That was a mess of a situation, and apparently, the vendor didn't think their configuration was part of their code.
That's an amazing viewpoint to me. The fact that any developer or manager thinks that their configuration data isn't a part of their code is worth testing. Yet, I see this attitude all the time, where developers, QA, managers, and more think that the code is the only thing that changes or doesn't change, ignoring the fact that there are configuration items that affect the code and need to be managed appropriately. Certainly, if the config data were in enums rather than in a file or database they'd feel differently.
Read the rest of Everything is Code
I get a tech newsletter most days, which has news that I enjoy, but interspersed among the news and ads are projects, frameworks, or repos, most of which I've never heard of before. I used to read these, but it seems that there is an endless list of these, which all have marketing descriptions that somehow claim this set of code solves problems that others don't or that this code is easy to use and integrate with, or well, I don't know what other promises. I'm usually turned off by the end of the first sentence.
The thing I've noticed is that there are so many projects out there. Even in the database space, if I happen to read a discussion on some aspect of databases, such as database deployment frameworks, I'll see links to technologies I've never heard of in my life. Some are small projects, and some are small companies, but there is an amazing variety of solutions for any tech problem. I'm not sure most of them are much different from the others, but the Not Invented Here syndrome seems to be everywhere. These observations also remind me of just how vast the world is and how little I see of it on a daily basis.
Read the rest of Keep. It. Simple.
If you have a data warehouse, what do you think your ratio of reads to writes is on any given day? Do you think 1:1, as in one read for each write? Is it 10:1, with 10 reads for each write? 100:1? Do you track this in any way?
One would think that most of the databases we work on in the transactional world have many more reads than writes. I'd have assumed the ratios might be higher for data warehouses, where we load data that is queried (read) as the primary use case. After all, I expect that there are lots of people querying data that is loaded into this warehouse, with relatively few changes.
Read the rest of The Load of Real Time Data Warehouses
When is the last time you read an article/blog/etc. on the Internet and saw a button for a print friendly version? That used to be something on every page, and one people often shared on social media (or email) because it didn't have all the advertisements in it. I remember having to help code this feature on SQL Server Central when we started as plenty of people wanted to print articles out and read them later. That desire led to Andy brainstorming that we should release The Best of books each year.
I was reading about how the Internet has changed many things in our lives and I thought about these links. I searched a number of places I visit often and there are no more printer links. I'm guessing with mobile devices and various save services, most people have gotten used to using digital technology to consume information?
Read the rest of Where's the Printer Friendly Option?
How many of you have objects in your database that aren't being used? What about something in a schema with a _old in the name? Or _2 or _3 or _delete? There is a lot of old, deprecated stuff I see in production databases. In fact, I've been somewhat amazed as I work with clients that many of the scripts we can build from a database with SQL Compare won't actually execute on an empty database because the script is full of broken code.
I also find plenty of DBAs that want to clean things up, but they don't. Sometimes they're afraid they'll break something, which is certainly possible. Sometimes they can never find the time. Often they might ask a manager, who usually says this isn't important and don't bother.
Read the rest of Cleaning Up Your Database
I'm not the smartest developer or DBA. I find myself mystified at times by Itzik's posts on T-SQL queries and I'm amazed at times by the complex systems that I see the DCAC people put together. I can usually figure things out (sometimes by asking the authors a question), but it's not always easy to do. We have some truly gifted, incredibly intelligent people in this business.
I am, however, effective. I have been very successful in my career at getting things done well enough, things that work well, meet the needs of my client/employer, and meeting deadlines. I don't just slap things together, but think about them, build them, test them (don't forget this), and then make sure they're working when they're deployed.
Read the rest of Effective Engineering
I saw a note from someone recently that reminded me of Policy-Based Management. This was (is?) a technology in SQL Server that I thought might have great potential. I even had a few presentations on the subject, but sadly I've rarely seen anyone implement it. I'm sure some do, but I think for me, this is dead technology.
There have been other tech items from which I've been turned off or abandoned over the years. It seems in SQL Server, we have some tech that even Microsoft has abandoned and doesn't put any development resources into improving.
Read the rest of Dead to Me
On a recent weekend, I got a text from my bank that they had declined a charge to one of my business accounts. I called them back and they let me know there had been a couple of weird charges on the account that their AI system detected. This seems to happen every year or two so I wasn't overly worried. I cancelled the card and ordered a new one.
A day later, my wife got a call about our credit card with the same issue. She cancelled the card and got new ones ordered. However, I use that card to travel and I had a trip booked. Suddenly I was without a credit. Luckily, we have another card for my wife's business that I could use. I called the bank and had a card expedited, but the situation created some stress. In fact, I panic-bought an RFID-shielded wallet. I've resisted for years, using an older, large wallet me daughter bought for me one Father's Day that always reminds me of her. The timing across a few cards was weird, and I suspect my wallet got scanned somewhere and both card numbers were stolen.
Read the rest of AI Is Great and Tech is Failing
Every once in awhile I hear about someone in law enforcement sure that tech people can build in a safe, secure way for data to be unencrypted by the company or vendor. The latest appears to be from Australia, where the Security Intelligence Organization wants tech companies to build this into products.
Backdoors never work. Anytime an encryption key is stored, it could be stolen. We see this all the time. Keys are just data, and companies lose data all the time. At scale. Governments are certainly not immune from this. One of the reasons that Azure allows a BYOK (bring your own key) for encryption mechanisms is that many organizations don't want to trust Microsoft to store their keys. I'm guessing Microsoft doesn't want the liability, either.
Read the rest of No Backdoors
I don't see a lot of SQL at The Daily WTF, but this one was great. It's a stored procedure that was likely just converted from embedded code, as noted by the poster. It's a strange set of code, that doesn't quite make sense to me, and I can't imagine why someone wrote it. Arguably, this is no better than having this code in a C# or ASP.NET application.
Or is it?
Read the rest of Bad Stored Procedures
"We need to get the code written for feature X. Can you finish this query today?"
We've all heard some variation of that request. We have a request or demand, and we need to get it done. We need to get code out so our business can advance, sell more things, get more customers, etc. There's always some reason to get new code pushed to production quickly.
Read the rest of When Do We Worry About Scalability?
There can be a big divide among tech professionals on how they view data privacy. Some don't worry too much, and some are very upset about the lack of data privacy and poor data handling practices from many organizations. Most people are probably in the middLe. I do know that every time I post about a Tesla, there are many people who mention the incredible amount of data Tesla collects, and how that makes them nervous.
Tesla discloses that they collect, and I get copies of my data with my own logger. I find the data interesting and I glance at it over time to look for trends. I'm still happy that my top destination in the last year (after home) is the gym ;)
Read the rest of Technology and Privacy
In general, I like simple solutions to problems (and simple songs). They can take work, but a simple solution means it's easy to explain to others, easy for others to refactor/modify, and we limit the amount of effort required to ensure others can use the solution. Sometimes I see very clever engineering solutions, but they become cumbersome to implement and maintain, or worse, no one other than the author understands how they work.
In any size organization, that's a piece of technical debt that can cause problems over time.
Read the rest of A Simple Solution
We are bringing back the SQL Server Central Opening Night party at the PASS Data Community Summit this year. This was one of the most fun things I've been a part of at the Summit for many years and I'm looking forward to the conference even more.
The party will take place Tuesday night, after the Welcome reception. from 8pm-10pm, once again catered as a casino themed party. I'm using the same company we've used in the past, and we'll have a cash bar and lots of prizes. I've already started to make a shopping list of fun things to give out as random prizes as you enjoy the games. In the spirit of competition, we will save a prize or two for the top chip winners.
Read the rest of Casino Night is Back
I saw a post recently about the incredible shrinking computer, where a software consultant had written down his predictions for the future. In this case, he writes about a computer shrinking smaller than today, essentially to one chip, with docking stations wherever you need them. We're already at the place where many of us have an extremely powerful computer in our pockets in the form of a mobile phone. I'm still amazed that I bought a phone recently with half a terabyte of storage.
A few years ago I gave a keynote where I looked at the changes in disk storage, starting with an IBM hard drive being loaded onto a plane. The capacity was 5MB. I remember working with KB floppy disks and MB hard drives, and lots of different connection technologies: IDE, ESDI, SCSI, and more. I've seen disks shrink from a 5 1/4" form factor to mere wafers with today's NVMe drives.
Read the rest of The Invisible Disk
On my new laptop, I only use containers as database servers. I made the decision not to install SQL Server or PostgreSQL and instead work on containers only. I've written lightly about this, but I set up docker-compose files to load different instances of SQL Server and PostgreSQL (and others) and batch files to start and stop them. I've also set dedicated places on my disk where I can drop backup files and access them from the host.
It's 2024. I moved to containers on my laptop exclusively for databases for the first time this year. This is despite the fact that I like containers, am comfortable with them, and find them handy. Moving from installed database server software to containers took a conscious effort, and it took time to configure everything. Really, it took me a bit of time to think about how I'd want to configure my system so that my work in SSMS went smoothly.
Read the rest of Container Development Work
Most of us will have more than one job in our career. In fact many of us will likely find a new job in the next five years. I hope I'm not in that group, but I recognize that it's a possibility. We never know when our situation will change, or our employer's situation will change. That is one reason I recommend you keep your resume up to date and continue to work on improving your skills.
I saw an office hours short recently from Brent Ozar, in which someone had asked him if they should apply for a job even though they didn't meet all of the requirements or know all of the desired technologies. Brent recommended the person apply, and his reasoning was that often a DBA (or other data pro) often gets asked to do a variety of tasks in an organization. The DBA job often crosses lots of boundaries and may end up working on a Active Directory issues, reporting, ETL, and more. When A DBA leaves a job, the organization looks for a replacement that can handle that same wide variety of things.
Read the rest of Fifty Percent
I had someone ask me about DuckDB recently. Would I think that's a good choice for a database. I don't really know. From their blog and some online research, maybe, but it's also a minority player in a niche space.
I had a chat recently with someone that had implemented ArangoDB, a graph database. Why that and not Neo4J I asked them? Someone at the company had tried it and recommended it. Not a bad reason, as I think experience with tech is important, but it's not the only thing.
Read the rest of Trying New Technology
I've heard of Kafka before. I know it's an Apache project and you can download or read more at https://kafka.apache.org/. I knew it was a way of moving data around, some sort of ETL tool useful for moving things around. More like a message and queueing system, which is a tool that seems like a great idea, but one that everyone struggles to work with.
And one that seemed complex. The overview is that Kafka is "a distributed system consisting of servers and clients that communicate via a high-performance TCP network protocol. It can be deployed on bare-metal hardware, virtual machines, and containers in on-premise as well as cloud environments."
Read the rest of A Kafka Introduction
I know a lot of people in this business do not have computer science degrees. While some do, I suspect it's a minority. I'm certainly curious, so if you want to share your education experience in a comment, tell me if you have a degree and what the focus was, as well as answer a few other questions.
At DevOps Days in Minneapolis recently, professors Fox and Sen from Macalester College talked about their computer science curriculum for growing the next generation of professionals. Along the way, they also asked the audience these questions:
Read the rest of Your Computer Science Education
A few weeks ago, I was sitting in the audience, waiting for my turn to speak at DevOps Days in Minneapolis. Just before me, Xe Iaso delivered a funny and thought-provoking talk on building a social network on a whiteboard. It was very well done and had me feeling nervous about following that session.
The talk is a bit of a satirical look at an interview Xe had for a company that tried to get them to derive an architecture for a large distributed system. It was interesting to hear Xe note that often we have architecture diagrams of what we'd like to have, but never an explanation of how we implement a large system, especially one that has to grow as our workload grows.
Read the rest of A Lack of Architecture and Planning
Many years ago I was training for a triathlon. I had competed in the Sandman Triathlon the previous year in Virginia Beach and wanted to do it again. I had a young child, work was busy, and I was struggling to find time to swim, bike, and run every week. One night, I was at a work event with a customer who was also a triathlete. He was much more competitive and successful than I was at competing in triathlons, and he told me I should just get up earlier and find time to train or ensure I spent time after work on training before I went home to ensure I was meeting my goals.
That sounded fine. Want to be better at something, then spend time on it. Certainly, that's what I often advocate for your career. Spend time on your career.
Read the rest of What Do You Drop?
I would hope most of you reading this know what SQL Injection (SQi) is and how you can prevent it. Or at least what patterns cause problems. If not, here's a short explanation that is worth reading. If you have more questions, ask in our forums.
SQL Injection has been, and continues to be, a problem in many systems. In fact, I chatted with Mike Walsh recently after he'd published this post on an attack for one of his clients. He has some notes that explain how your database server might be vulnerable, as well as a description of a recent attack example. He also notes that many of you are responsible for protecting data, which is separate from other security mechanisms. You need to be sure you are protecting your data, even in vendor applications.
Read the rest of Prevent Complete SQL Injection
One of the challenges many people have is focusing their learning efforts along some path. The best way to move forward is with steady effort that guides you through steps to build knowledge or skill. However, with so much information out there on the Internet, how do you decide where you focus your efforts?
Lots of people choose a random method, but the world is full of those people, many of whom never develop strong skills. That might be fine if you are an hobby guitarist or piano player, but it's not the best way to approach your career.
Read the rest of A Checklist for Learning
This is part of a series on observability, a concept taking hold in modern software engineering.
One of the interesting things I saw in an engineering presentation on Observability from Chik-Fil-A was that they are sometimes bandwidth-constrained at remote sites. In an early version of their platform, they sent logs back to HQ, and their logs used all the available bandwidth, so they were unable to process credit card transactions.
While most of us don't deal with lots of remote offices sending data back to a central data warehouse, we do often work in distributed environments, and we may send data to/from a cloud or even employees' remote offices. Or maybe we send a lot of data between components. Bandwidth is very good in many parts of the world, but it isn't infinite.
Read the rest of Send Metrics Not Logs
Ryan had a planes, trains, and automobiles situation a few weeks ago. On the same day, I didn't. A delay for a few hours, but an easy trip home. Lots of other people didn't have a smooth trip, and I had a few friends who spent an extra night somewhere or had flights canceled and decided not to take a trip. The Crowdstrike outage hit the entire world, causing problems everywhere. Just before that happened, Azure had an outage in the central region.
If you were affected by these, you have my sympathies. If your IT job included responding to these, you get all the virtual hugs from me (and Brent). I know what it's like when someone calls you to handle an outage, and I know it can turn your life upside down. Hopefully, it hasn't been too stressful to you.
Read the rest of Responding to a Disaster
I visited a customer last week and attended SQL Saturday Baton Rouge 2024. Both were fun events, and an enjoyable week, though I was away from my wife for 4.5 days, which she didn't love. Today, I had a quick turnaround, heading to Wisconsin Dells for THAT! Conference, which I attended last year and enjoyed. I didn't submit this to the event, but got asked to go as part of my job. I accepted, and I'm gone from home for a week between these two trips.
I do travel a lot, but these trips got me thinking about how many of us might handle the unexpected demands from our companies. In this case, I had planned on SQL Saturday Baton Rouge, but not a customer visit. I got asked a month ago to add this in, which was fine, but two weeks ago the trip got extended by another day. Just before I got the update from that call, I agreed to go to THAT! for a quick presentation.
Read the rest of A Quick Turnaround
Technology has dramatically changed the world over time. The advent of cars dramatically changed the US, as people could go places and meet others in a way that was difficult and slow before. The telephone let us communicate with people all over the world at a pace that was previously impossible. Computer technology has furthered this at a truly amazing pace, especially since the adoption of mobile devices by so many people. The flexibility in how we can integrate computer technology into our lives has been incredible.
However, each technology change brings about plenty of negatives and potential problems as well. I ran across a piece from L. M. Sacasas that has some questions we might ask about any technology, including the software we build. The start of the piece is that most of us don't think about how our work might be misused, which can lead us to dismiss security risks or moral misuse risks. We often don't consider the malicious ways people view applications.
Read the rest of Thinking About Technology
Today I have a question for you:
What are the three biggest challenges you face today as a database professional?
Read the rest of The Biggest Database Professional Challenges Today
I caught this piece on the need for programmers (developers) to not trust anyone, including themselves. It is written by a software developer for other software developers, but I think it can also apply to database work as well. It is a bit long, but it starts with the nature of abstractions in the world and how they let us work with simpler models of a situation or environment. However, most abstractions are leaky, and our assumptions about them can cause our systems to fail.
The leap from trust to abstractions seems a bit funny, but it makes some sense. We ought to simplify our situations so that we can generalize how to solve them, but we also need to verify things. There are a few examples, one of which is we ought to use tests to ensure the code does what we think it does, including using a wide variety of data. We ought to ensure that refactoring something doesn't break the system, or more often for databases, we return the same results. Changing a query to run more efficiently with joins or a window function instead of a loop or subquery should return the same results. Not just for one row but all rows, and across different inputs.
Read the rest of A Programmer's Attitude
The cloud has been a controversial concept for much of its existence. While the idea has been around for many decades, AWS started selling IT services in 2006, with Azure following suit in 2008. Since then, the use of cloud services has grown tremendously. While some applications and organizations have embraced the idea from the beginning. I found many of you at SQL Server Central were very hesitant at first. I guess some of you are still skeptical about the value of a production database in a public cloud.
From the beginning, I've felt that cloud computing has a place in the world, but in a way that is more appropriate for some situations than others. In terms of database (and maybe compute services), if you have a very well-known and predictable workload, the cloud can be very expensive. It might still be a good choice, but I think it often isn't. If you have a variable or growing workload, then the cloud might serve you better than trying to keep up with new hardware in your own data center.
Read the rest of Cloud vs On-premises
I assume that most of you know about the principle of least privilege. If not, please read this short blog from Brian Kelley and make sure you understand how you should approach security. In the modern world, we also ought to adapt our systems for the zero trust model, which includes the least privilege principle.
However, I wonder how many of your organizations really follow these security guidelines internally. Are you strict about adding limited access and removing it when people change jobs/roles? If you use Windows Auth (or Entra), are your admins doing that or just adding in new roles? Do you scope down database access roles in granular ways or just stick with 1-2 roles for the most common things people do?
Read the rest of Least Privilege
Stretch Database is finally going away. It is being retired. It was deprecated on Nov 16, 2022, from SQL Server 2022. Effective Jul 9, 2024, the supporting Azure service is retired. I saw this in an announcement on Jul 3, though I hope anyone using this service has been seeing lots of reminders over the last couple of years. I know I'm getting MySQL retirement notices for one of my services and need to migrate some workloads this month.
If you tried this service, you might have realized that the pricing didn't make sense for most of us. If you hadn't tried it, it worked by moving some of the data in your tables into Azure, where it could be queried if needed. It was an interesting idea, though most of us would have wanted this to work between two SQL Server instances, not between SQL Server on-premises and Azure.
Read the rest of The Stretch Database Retirement
I saw a blog post from Randolph West recently that asked How do you restore a SQL Server 2000 database in the year 2024? It's a bit of a process, involving an intermediate version and two restores. He also points out the need to run DBCC after the first restore, which is a good idea. I wonder how many people would take the time to do this, or even think about it as an upgrade step?
This was interesting to read as I had a customer ask me about doing this a few months back. They were trying to clean up their database estate and modernize some of their older systems. This was becoming a big project for them, as they had several pre-2017 systems, none of which were in support. Auditors, regulatory authorities, and even business partners see this as a large security risk and get concerned if you're running older software.
Read the rest of An Upgrade Slog
Yesterday was Independence Day in the US and a day off for me. Today, I'm back to work. Same for my rocket engineer son, one day off. My wife remembers a few of her companies giving a 4-day weekend when a holiday falls on a Tuesday or Thursday, but that hasn't been my experience and isn't this week.
I know we're barely into summer, which technically started Jun 21, but most of my life has measured summer between the end of one school year and the beginning of another. That's usually late May (Memorial Day) to early September (Labor Day). I think that's a very American thing, as I know lots of my friends in Europe are just starting their break between two grade years. For those of you in the Southern Hemisphere, I'm sure this time of year is even more different.
Read the rest of Midsummer Fun
I had someone ask me recently how to run xp_cmdshell on a Linux version of SQL Server. I told them you can't, as it's an unsupported feature and not one that I expect to see released. I had to double-check, since I did think that supporting a BASH shell was a possibility, but it wasn't added to the product.
In the feedback forums, I saw a request for xp_powershell, though the feedback from MS is to use CLR for this. They suggest external access permissions, but those aren't supported on Linux. I also didn't see a request for shell scripts added, and I'm not sure I want one.
Read the rest of Xp_cmdshell Use Cases
I often deal with customers who are looking to improve the way they build and manage database software. These could be small companies or large enterprises, with teams of developers trying to enhance their application software to solve new business problems. Often those enhancements require new data, with the related schema changes in a database. Even if there isn't any new data, often we need to query data in new ways, combing, filtering, aggregating, and otherwise transforming data into extra information that a business can use.
Solving many of these problems is iterative by nature. We try one thing, then another. Often a developer might experiment with a data model or query, trying to match a requirement they've been given. Once they produce a solution, we may find problems in testing, or far too regularly, in production. These could be data-related issues, where the developer hasn't considered values in their solution (zeros, blank or long strings, extreme dates, etc.). These could be logical errors, where the developer just made a mistake. There could also be a problem with the requirement, where the customer provided an incomplete or incorrect specification to the developer.
Read the rest of The Cost of Rework
One of the hot terms in software these days is observability. There are a few definitions (Splunk, RadixWeb), but essentially this is the insight into how your software runs and performs using metrics, logs, traces, etc. In DevOps, we do this with an eye toward improving performance and identifying the root cause of issues. The focus is slightly different from monitoring, where we often focus more on resources and health. We need both, but often in trying to improve software and the behavior for users, developers need observability. Infrastructure people responding to acute issues and looking to ensure we have the capacity, availability, and other x-bilities, that need monitoring.
Today I'm wondering if you collect a variety of types of metrics for your software that might tell you how your system is running. What things are important to you in order to better serve your clients? If you're a DBA/sysadmin, what is important to you? If you are a developer, are there different types of data you want?
Read the rest of What Metrics Do You Collect?
The last 10-15 years in software development have seen a widespread embracing of unit testing. Before the popularity of mobile phones and their apps, most of the organizations I'd worked in gave lip service to automated unit testing, and often even more complex integration/system tests.
These days, it seems more and more people embrace unit testing, and I hear about that often from customers and attendees at events. I don't often hear about more comprehensive integration and system testing, I found an interesting article from the Pragmatic Engineer that looked at how the Bluesky social network was built. The article is partially paywalled, but I have subscribed because of the interesting thoughts they publish. In this article, there was a really interesting part of the article on testing. This is a section titled "Integration tests over unit tests".
Read the rest of Where to Test Your Code
There are lots of resources for learning: articles at SQL Server Central, blogs, user groups, SQL Saturday and other events, conferences, and more. In most of those cases, the editor, author, or speaker is deciding what they want to write about. If you want to learn something different, you need to go search out that information. You can certainly request topics from others, but they may or may not listen to you.
At least not as an individual.
Read the rest of What Do You Want to Learn?
In the last few months, I've been traveling around at a few of the Redgate Summits (one more in NYC coming) running panels on cloud journeys. I've had industry experts, both technical and managerial, discussing their approaches and journeys with advice and caveats for others. It can often be more than just migrating systems, so a lot of people have started to talk about cloud transformation.
However, in some cases, this is just a migration. A lot of companies just lift-and-shift their databases into the cloud, along with various other services. While this is a quick way to get into the cloud, it isn't much of a transformation. If you review and right-size the resources you've provisioned, maybe there is a bit of a transformation, but not a lot.
Read the rest of Cloud Transformation
SQL Injection has been a problem for my entire career. Thirty years ago I could have easily just blamed this on ignorance, as most of our developers didn't think about the nefarious ways that hackers enter data in our applications. These days, there isn't a good reason for this to keep happening, and the problem is us. I think that we don't provide good examples or training on secure coding or secure architecture as a normal part of teaching programming. In many organizations, we don't check for issues and prevent their release. Some do, but many don't. On top of this, the existing code is usually a poor template for writing future code. I do think Microsoft aims for secure coding in SQL Server but in Windows, there is work to be done there.
A few months ago, I saw an article that noted the US CISA organization and the FBI issued a secure-by-design alert (PDF) that noted there is no excuse for SQL Injection vulnerabilities (SQLi) in modern software. This alert notes that SQLi has been an "unforgivable vulnerability" since at least 2007. Inside the document on vulnerabilities, it notes that a single quote can't be used in certain fields: username, password, ID field, or numeric field. They also note that co-mingling user data and query data, like constructing queries on demand, is a poor practice.
Read the rest of SQL Injection Is Not Acceptable
I almost called this "chasing a new laptop" since that's what I'm doing, but I decided to add the date because the current laptop I've using was built in March 2019 and got to me in May 2019. I've had an HP Spectre x360, my second HP Spectre, and I've really enjoyed it. I'm also amazed it still runs. On the last few trips, the two rubber strips that run along the bottom (acting as feet) started to peel away. I've never seen that before and I tried to re-attach them a few times, but that didn't work well.
Not a big deal, and I can live with that, but then during my Australian tour, the laptop started pausing and freezing a few times. It might be that there is too much software on there and needs a pave-and-reinstall, but I decided to check the age on the machine. That was when I realized it was five years old. It's been a great machine, but I don't think I've ever had a work laptop last that long with daily use. Of course, there was about a year during the pandemic when it was rarely used, so maybe its life lengthened during COVID.
Read the rest of Five Years and Counting
One of the interesting things that I see at Redgate Software is how idealistic our developers and engineers can be. They often build our database DevOps products with the idea that customers will use well-designed databases. The systems will have primary keys, foreign keys, defaults, constraints, indexes, and more. Developers will use coding standards, and naming conventions, and will understand what data is stored in tables. Not in every case, but often.
After all, that's how we build software, as teams, sharing information, publishing documentation for others, and following best practices.
Read the rest of Poor Database Design Realities
At the Redgate Summit in London, I ran a panel talking about Platform Engineering and how we can make developers more productive. One of the questions from our audience revolved around AI (Artificial Intelligence) technologies and how they might assist. As a note, AI tech includes a lot of different things, like machine learning (ML) among other things, but a lot of people seeing the media and hype around LLMs (large language models) think those are AI. They see AI as what is implemented in ChatGPT and Copilot, which is correct, but incomplete.
One of the panelists, Jeff Smith of Redgate, said that he views the output from AI as a first draft, something that bootstraps further work by a human. This can save time and can help someone be more productive, but it's a starting point and a boost, not a final product.
Read the rest of Using AI for the First Draft
I saw a tweet that DBCC CLONEDATABASE was being discontinued for production databases, which both scared me and didn't make sense. I've used this a few times for a quick copy of a database and like how it works. Discontinuing it seemed strange to me.
Then I read the blog post, which notes that it's not being supported for production deployments. The post doesn't explain why, but I'm guessing this is because all the stats and other metadata moves, and this might mess up the optimizer if different data is added. I don't know who deploys production databases like this, but I could see people who have federated or sharded databases using this to create a new blank copy and then uploading data into it. Or maybe people who need new databases that are distributed onto remote office/edge devices used it? If you use this to create production dbs, let me know.
Read the rest of DBCC CLONEDATABASE is Semi-Discontinued
I have worked for a few startup companies, including SQL Server Central. Each has been a different experience, and I learned a lot at each stop. However, I'm not sure I'd want to go through that process again at my age. I was thinking about the challenges and the excitement of being at a startup while reading about the founding of Reddit. The post doesn't go a lot into the technical details or the working life, but it is an interesting read from a VC investor.
I also found this post on Choosing Startup Life, which talks about what the author thinks about before trying to start a company. He compares this with life in a Big Tech company, which relates to lots of companies, in technology or not. The main differences are lower salaries, less infrastructure, lots of work, and upside in a startup. Big companies have higher salaries and more perks, less stress and responsibility, and not a lot of context-switching. In general, that's been true in my experience, though in bigger companies that didn't think they were software companies, I sometimes could end up with a lot of context-switching.
Read the rest of Life in a Startup
When we build software, many of us use the same algorithms to solve problems. We might choose a similar method for a quicksort or a lambda validation or a regular expression. For database work, your code for a running total (or other common challenge) is likely very similar to many other people. At least on the same platform. You might solve this differently in SQL Server and Oracle, but for the same type of database, many of us write very similar code.
Actually, for many developers, they copy and paste an answer from SQL Server Central, Stack Overflow, or another site. I'm not sure if I think this is good or bad, as it's a good idea to reuse code if it solves the same problem. If you copy it and don't test it, that's bad. After all, the code might not solve your slightly different problem if you don't check it.
Read the rest of Common Algorithm Concerns
Several years ago, I heard about a new product coming in Azure that would provide an IaaS (infrastructure as a service) VM to run SQL Server, with Microsoft managing most of the admin tasks for the instance, like patching and backups. That didn't seem like a big load to me, and I wondered if anyone would actually pay for this product. After all, don't most companies find managing patches and backups?
That product became Azure SQL Managed Instance, and I've been surprised at the adoption. Quite a few clients have adopted this as a way to lift and shift (mostly) to the cloud in an easy fashion without the restrictions of Azure SQL Database. This looks like a "normal" on-premises SQL Server, and there are both high-performance (Business Critical tier) and average-performance (General Purpose tier) versions of the product that let you choose what level of price/performance you need to achieve.
Read the rest of Managed Instance Impressions
Most of the people I know who speak at a SQL Saturday or user group aren't paid for their efforts. At many of the community events, the speakers are volunteering their time. Many are also paying for their own way to those events not located in their area. A few, such as me, might get a company to cover their travel expenses, but often this doesn't include time. If I attend a SQL Saturday, I still have a bunch of work on M-F that needs to be done. No comp time for these events. That being said, I'm happy to donate some time and money to community events.
Some speakers build and teach full-day sessions, usually called pre-conference sessions, for which they are paid. There can be competition at large events like the PASS Data Community Summit and SQL Bits to get a session since the payment can be rewarding. I've seen some speakers make USD$1k or so, which can cover travel expenses, and others make over USD$10k, which is a nice payday.
Read the rest of The Costs and Rewards of Speaking
This article has a concept I've never heard about: invisible downtime. This is the idea that there are problems in your application that the customer sees. Your servers are running, but the application doesn't work correctly or is pausing with a delay that impacts customers. From an IT perspective, the SLA is being met and there aren't any problems. From a customer viewpoint, they're ready to start looking at a competitor's offering.
Lots of developers and operations people know there are issues in our systems. We know networks go down or connectivity to some service is delayed. We also know the database gets slow, or at least, slower than we'd like. We know there are poor-performing code and under-sized hardware, running with storage that doesn't produce as many IOPs as our workload demands. We would also like time to fix these issues, but often we aren't given any resources.
Read the rest of Invisible Downtime
Kubernetes is cool, and I think it's really useful in helping us scale and manage multiple systems easily in a fault-tolerant way. Actually, I don't think Kubernetes per se is important itself; more it seems that the idea of some orchestration engine to manage containers and systems is what really matters. As a side note, there are other orchestrators such as Mesos, OpenShift, and Nomad.
However, do we need to know Kubernetes to use it for databases? This is a data platform newsletter, and most of us work with databases in some way. I do see more databases moving to the cloud, and a few moving to containers. I was thinking about this when I saw a Simple Talk article on Kubernetes for Complete Beginners. It's a basic article that looks at what the platform consists of, how it works, and how to set up a mini Kubernetes platform on your system. It's well written and interesting, but ...
Read the rest of Kubernetes is Cool, But ...
It seems that when I travel to offices these days, it's standard for most desk setups to have two monitors. I think all the desks at Redgate have a docking station and two monitors for people to use. They also convert to standing desks, which is handy. I have a standing desk that I use regularly, and it's nice to have that option when I visit an office. At a number of customer sites, I've seen similar setups, sometimes with laptop/monitor lifts instead of desks that rise.
Recently I saw a docking station announced that can support four monitors. I wonder how many of you want, need, or use more than two monitors. While there is often a standard in offices, since many of us work part or full-time at home, perhaps you have a different setup. Maybe you have one or two large monitors instead of 3 or 4 smaller ones. I'd certainly be interested to know if any of you have more than 4 monitors.
Read the rest of Multiple Display Productivity
I've got a few certifications and quite a few more that have expired or aren't relevant. Does anyone think Windows NT 4.0 or SQL Server 6.5 matter? If you need help in those areas, ask someone else. Unless you have a crazy budget with a willingness to pay a ridiculous hourly rate.
Kamil Nowinski had a recent video discussing why IT certifications are still relevant. He had ten reasons, and if you want to watch the entire show, you'll hear his reasons and some rationale why he thinks they matter. It's a good set of reasons: keeping up with tech, practicing learning, demonstrating a commitment to some technology, finding a community of certified colleagues, and more.
Read the rest of Are IT Certifications Still Relevant?
One of the things that I've found with cloud computing services is that the people involved with managing these resources sometimes get asked to become financial accountants.
I saw an interesting post from SQL Rod asking about this new task as something all of us might need to consider a part of our job in the modern world. He asks if we are Techouncants or Accountechs? This isn't in the sense of being the Financial DBA trying to manage costs in the cloud, though that is part of his post. It's more about making smart financial decisions. Certainly, as more workloads move to the cloud, and they are for many of us, we likely need to keep an eye on costs, usage, and tuning.
Read the rest of The DBAccountant
For most of us working in technology, I think we understand that if something is broken we might need to work. Not that we have to, or we need to, but we might need to. Perhaps you feel differently, or your company approaches on-call in another way. If so, let me know today how you deal with staff being on-call.
In my career, there are jobs with formal on-call, informal on-call, or even no on-call. In the latter situation, there isn't anyone who is prepared to handle issues outside of normal working hours, but that doesn't mean if management calls you can ignore them. It's that the organization didn't expect issues. I worked in a small company (< 50 people), where we primarily had systems for people who worked in the office, and nothing was running at night (outside of backups). Normally no one knew if there was an issue overnight or on weekends, but I did get called by the owner when he went in one weekend and couldn't receive a fax on our computer system. So I guess I was the emergency-on-call person.
Read the rest of The On-Call Load
I saw a quote recently that resonated with me. It's not something I've often struggled with, but I have at times. Here's the quote:
"Life rewards action, not intelligence. Many brilliant people talk themselves out of getting started, and being smart doesn't help very much without the courage to act. You can't win if you're not in the game." - @JamesClear
Read the rest of Action Over Knowledge
Last year, I read Surrender, a book by U2 lead singer, Bono. Bill Gates listed this as one of the top books to read at one point, so I picked it up and dove in. I have enjoyed U2s music since I was in high school, and was interested to hear what made Bill Gates recommend his book. The book is partially a journey of U2, but mostly a look at how Bono's view of the world and life has changed over time.
Bono grew beyond music in his life to become an activist and try to shape the world into a better place. Whether you agree with his efforts or focus or not, it's admirable that he has tried to be more than a rich and famous singer. He's had to build more skills around how to communicate with others, convince them to take a course of action, and educate himself about the world. In trying to build these skills, he's founded or worked in organizations around his time with U2.
Read the rest of Seagull Management
Recently I traveled to visit a customer who has an in-the-office culture. They have multiple large buildings outside a major US city and almost all their employees (7000+) live nearby and are expected to be in the office the whole week. More senior people can opt for 4 10-hour shifts rather than 5 8-hour shifts, but with few exceptions, they have people in the office.
I hadn't seen that in a long time. Almost every customer is mostly remote or some level of hybrid (usually 2-3 days a week in the office). What's more, they have an open culture, with rows of desks for teams and spaces between the rows for managers and directors. No cubes!
Read the rest of Missing the Office
As I work through 2024, I found myself doing a little more vacation planning this year than in previous ones. In 2022 I traveled quite a bit, but my wife went with me often. We went to Europe 5 times that year and added quite a few vacation days around my work trips. My wife thought that was a great year.
Last year, 2023, was different. I traveled more (36 trips), with most of them being short. When I traveled that much, I wanted to end trips quickly and get back home. I learned that was too many, and also too disruptive for life. I got behind on things I needed to do at home, my wife went with me less because many trips were all work, and I lacked energy from the pace of moving all over the world.
Read the rest of When Are Your Breaks?
One of the ideas behind DevOps is that we minimize the time between code commit and deployment to production. We want to avoid work-in-progress and bottlenecks to getting our software into the hands of customers. This has led a lot of companies to release more often, albeit with smaller sets of features. The total number of things delivered under DevOps might not be greater, but it often is more targeted to those things our customers want/need/use.
However, the idea of releasing often means that we try not to stack up too much work before deploying it. What does that mean for holidays and the code freezes or no-deploy periods that many companies have? How do you implement a code freeze under DevOps?
Read the rest of The Code Freeze
I subscribe to quite a few newsletters, and one of them is for job hunters. I have a great job, maybe the best job for me, but I like to keep in touch with what's going on because I, well, I have job insecurity and worry about being without a job. I think this is leftover from childhood. I also want to know how to help others in finding their dream job.
Recently there was a question asking when it was time for a new job How can you tell (for yourself) when you should be seeking new opportunities? It might not be easy to recognize for many of you. Perhaps you don't realize there are other opportunities you might appreciate because you're comfortable. Perhaps you're the frog in water that's being heated so slowly that you don't realize how poor your current situation is for your health.
Read the rest of When is it time for a new job?
I've had a good career in database work. I've had success, and I've had some failures, fortunately the former far outpacing the latter. In my career across many companies, the code I've written has tended to work well, or at least well enough. I've managed systems and ensured a high uptime, and solved issues quickly. I have left quite a few jobs in technology, some because I was unhappy, some for better opportunities.
I was asked to leave one job. I disagreed with my boss, thought he was a jerk, and our CTO told me this person was more valuable than I was at that time. The CTO suggested I move on, so I did. That day.
Read the rest of Under the Bus
CosmosDB has been a great data platform in the Azure cloud that helps companies deal with disparate types of data. The CosmosDB APIs include those for MongoDB, PostgreSQL, Cassandra, and Gremlin. These wire-level protocols let you work in a way that is compatible with those systems for storing data.
That's coming to SQL Server.
Read the rest of CosmosDB APIs Inside SQL Server 2024
I assume most of you reading this work with SQL Server, at least for some of your workday. I know there are plenty of you who also support Oracle, MySQL, PostgreSQL, or some other database platform. The results in our (Redgate's) State of Database Landscape report showed that many organizations, indeed most, have more than one database platform in production.
This was also a theme in our Data Community Summit and Redgate Summit keynotes, where Ryan and Grant discussed their journey to learn a new platform (PostgreSQL). One, a requirement (Ryan) for a new job, and another, an opportunity (Grant) as the company focus shifted. I assume some of you out there have had similar experiences either moving towards, or away from, SQL Server.
Read the rest of The Journey to Change
There is a skill that I think DBAs and sysadmins will need to develop: cloud cost analysis. I've thought this was important for quite a few years, and I've been (unsuccessfully) lobbying for cost information to be gathered and analyzed in Redgate Monitor. Hopefully, this work will get done soon, as I see more companies asking their technical people to provide analysis and justification of the resources being billed for in the cloud.
Basecamp analyzed its costs in 2023 and decided it could save money by leaving the cloud. I've seen other companies decide they were saving money in the cloud. Many, however, are likely unsure of the total return they get compared to the costs of cloud computing. I have seen some posts (like this one) that try to help you get a handle on your costs, but there is often a lot of complexity in cloud costs when multiple departments have different accounts (AWS) or subscriptions (Azure) with a provider.
Read the rest of The Cloud Database Cost Analysis
Chocolatey Solutions Engineer Stephen Valdinger said, "DevOps isn’t something you do, but rather, it’s a way of doing things. What works for us here, may not work for you there, so you adjust." He then went on to say that DevOps is a way of working that reduces time to introduce changes, while at the same time making changes traceable, accountable, and revertable.
I've seen many companies try to copy what another company has done, especially with regards to DevOps and software development. I see companies copy the organization of teams from Amazon, Spotify, or others. Often quite a bit of time and effort is spent changing the way your development team works, and often without a lot of success.
Read the rest of Another View of DevOps
I wrote a post about Git recently since I've found far too many people who didn't know Git well (or at all) and they seemed hesitant to experiment and learn.
Is this because of being too busy? Do we have so many people who are working in tech where "it's just a job" and not because they enjoy technology? Or has management in too many places beaten people down to the point they don't want to try things? Perhaps your coworkers are resistant to change (or lazy) and you don't want to discuss new ideas with them.
Read the rest of An Experimental Mindset
Re-platforming is the process of moving a system to a new platform. Imagine taking an ASP.NET/SQL Server website and moving to Azure Functions on Azure SQL Database. Or maybe taking a Java client/server app with Oracle and moving it to a series of microservices against MongoDB. Those changes could be a net benefit to your organization in the end, but they aren't quick or easy. They're often fraught with various challenges that can cause a lot of stress while creeping over budget.
There's a post that talks about some of the things you might think about if you embark upon a re-platform. Often this takes place when an organization is looking to modernize their tech stack. Quite a few of the technology DevOps success stories take place when the older structures are not maintainable, but also not able to handle increased workloads or performance requirements.
Read the rest of Re-platforming
Legislation was introduced in Australia to allow employees to protect themselves from unreasonable calls and emails from work during off-hours. You can read more about it, but a bill was passed to allow employees to ignore calls and messages without repercussion It is expected to pass, though I don't expect this to do a lot for bad bosses. They'll find ways to hint or imply that you need to respond, and might even remove your chances of advancement/promotion/raises. While you can sue, that rarely works out well for anyone.
From the various reports I've seen, I'm not sure if this includes salaried workers or just hourly workers. I certainly think hourly workers ought to be paid if they're called after hours. I've even been paid as a salaried worker if the calls outside of core hours exceeded a certain amount per week. That seemed fair to me, and in our team, there was always someone willing to work more for extra money, so I could trade on-call work if I didn't want to do it.
Read the rest of Protecting Off Hours
Redgate released the results of their State of Database Landscape recently, and I was part of a webinar where Beca Parker, Ryan Booz, and I discussed the results. As we were talking, one of the things that struck me was the high number of people who had picked skills as being a challenge for their organization. In a single environment, skillset requirements and training were the number one challenge, and it was listed as the number one obstacle to implementing a DevOps approach.
While this industry is full of self-taught individuals who have spent time learning new technologies and tools, it's somewhat amazing that training continues to be an issue. Many organizations limit their training budgets and time allocated to employees while continuing to expand the number of technologies and platforms they use. This is one reason why I think changing database platforms to avoid licensing costs is unlikely to save you money anytime soon. Retraining staff and developing competence takes time. Perhaps this is also one reason why many companies look to the cloud, thinking that they can reduce the amount of upskilling needed by their staff if the cloud vendor manages the systems.
Read the rest of Growing Skills at Work
The title of our keynote session at the Redgate Summit in Atlanta is Navigating the Database Landscape, and I'll be delivering part of the talk, along with Grant Fritchey and Kathi Kellenberger today, Mar 13. This is based on the State of Database Landscape Survey results, as well as our experience working with customers and implementing DevOps solutions over the last decade. The talk was mostly written by others, but as I rehearsed the session, I found myself wondering about how I'd approach my job if we returned to being a DBA or developer.
When working in technology today, there are many challenges outside of actually learning about any of the particular products, languages, platforms, etc. We have the politics of working with others, ongoing work, emergency requests outside of channels, random questions asked by others, code reviews, and probably a few other things I'm forgetting, all outside of learning any new skills. While I consider myself a lifelong learner, I know that finding time (and energy) to acquire the basics of any new technology is challenging.
Read the rest of Navigating the Database Landscape
I saw this on one of the syndicated SQL Server Central blogs: Job Description of a DBA. This is mostly a list of things, but it covers a lot of the general types of things that many of us expect a DBA to do. However, my question is whether you think this is a good description for a job opening. Would you ask someone to fill this list of things?
Maybe a better question is whether you could use this to evaluate candidates, and how would you go about asking questions or gauging a fit based on this. Perhaps you should ask yourself how you evaluate others and if there is a good way to rate them against a list of daily tasks.
Read the rest of Is This a Good DBA Job Description?
When I started working for a living, there were secretaries in many organizations. These were people who actually did a lot of correspondence (written or verbal) and busy work for managers or executives. Over time, as email and computers became commonplace on desks, I saw fewer of these positions. As more people started to send email, we had to actually alter software to allow assistants to impersonate their bosses and manage the volume of communications that many of us deal with.
We're in a new age of assistants with the emergence of Generative AIs powered by LLMs that can appear to respond in a conversational style to requests and perform actions on our behalf. In this new era, will AIs function as old-style secretaries, handling simple, but important tasks? Are they the trusted helpers that secretaries used to be for many executives? Are we all going to have an assistant, and do we want one, or need one?
Read the rest of Assistants in the Age of AI
It can be hard to find candidates for positions these days. I know there are a lot of people looking for jobs, but are they the ones you want? Are they a good fit for your team? Those are hard questions to answer when culling through resumes, conducting phone screens, and sitting in interview rooms asking questions from a template. What can be even harder is to compare different candidates when multiple people may interact with the candidates. Even if you do all the work, you're busy. Can you make good comparisons of different individuals that you've spoken to across a few weeks, in between all the real work you're trying to get done?
I know I've struggled to do this, no matter how many notes I take or how long the discussions are with other team members. I have had a very mixed bag of success in hiring.
Read the rest of Creating Candidate Interest
AI (Artificial Intelligence) systems and technology has been all over our industry for the past year or so, ever since ChatGPT released the initial public version in late 2022. It seems that there is a lot of hype around the possibilities, with plenty of excitement and skepticism, depending on who is talking about the tech. However, there do seem to be some places where the technology is working well, and security is one of them.
There is an article about how Microsoft is using AI to help spot ransomware, which seemed to have run rampant a few years ago. It's still around, though it seems fewer exploits are being publicized. That might be because systems are better protected, perhaps there are fewer attacks (unlikely), or maybe more organizations are getting better at covering up their issues. They might be better prepared to restore backups or quicker to pay a ransom.
Read the rest of Using AI for Security
The inspiration for this was a piece about the art of the good code review. Throughout my career, I've seen code reviews grow and change. From formal meetings to automated notifications and asynchronous discussions to complete lip service to the process. I'd like to think that most organizations are beyond the latter and there is some sort of review beyond the developer, but I still see a lack of other eyes looking at code before it's deployed, especially database code.
The article above opens with the idea of why we review code. The main reason is to create ownership, or more specifically, shared ownership. I had never thought of it in these terms, even though I think the ideas of standards and patterns are certainly shared items. Having everyone take ownership not only keeps quality high but could help you share knowledge and also ensure everyone feels a responsibility to safeguard all the code. This also helps everyone keep an eye on the larger picture of the entire codebase.
Read the rest of The Art of the Code Review
About ten years ago my sister-in-law broke the screen on her mobile phone. She'd had an older iPhone and when she went to upgrade, none of the upgrade processes worked because her OS was so far behind that they couldn't transfer her information smoothly. She had been avoiding OS updates because they interrupted her life, but that was now a problem because the world had marched so far beyond her version that there weren't tools, or at least, no one was interested in trying to perform an upgrade across multiple OS versions (I think it was 3 or 5 versions).
I ran into this recently with someone else I knew, but not for a mobile phone. For TFS 2015. This customer had been working along with this older system and is finally ready to upgrade to Azure DevOps in the cloud. They wanted to know if they could somehow upgrade the TFS database and move all that data easily into the cloud. I said this wasn't likely easy as this isn't an upgrade, but an export and import of a lot of data. Microsoft offered a path, but it was multiple upgrades before an export/import, which was deemed too expensive. Right now, I'm not sure what they're doing to do.
Read the rest of The Dangers of Not Upgrading
No, I don't have a way to license a SQL Server instance for your application for free. Microsoft still charges for production workloads. However, there are several ways that you can do development or testing for free, or at a greatly reduced cost. Bob Ward spent a bit of time researching options and he's written a post that outlines a number of your options. It's very comprehensive and I know a lot of us are grateful to Bob for summarizing our options.
As I work with developers in organizations, there are often questions about the licensing for dev editions. This comes up more with Oracle than other platforms, but SQL Server does get asked about. We made a decision to support SQL Server Developer Edition in Redgate Clone, which is in line with what Bob's post above says: this edition is for developers (and it's free). There is one caveat here, which I never knew. The licensing guide says you cannot build test data and move that into production. I'm assuming this doesn't mean you can't create a list of US states, countries in the world, or statuses. I think this means something more substantial, but if anyone knows, let me know.
Read the rest of Free SQL Server
I drive a Tesla Model Y and love it. They're not for everyone and not for every situation, but I think it's the best car I've owned. My wife feels the same way, and if we had a need for two cars, we'd likely get another one. The car is fun, and I look forward to getting in and moving around town. If you have questions about owning an EV or a Model Y, please feel free to ask.
The car isn't perfect, and there are things I wish I could change. There have also been a few recalls on the car, which is interesting for a Tesla. A few of these recalls meant that I got a software update a few days or weeks later and something changed. There was another recall recently, which affects most Tesla vehicles in the US. I'm sure that by the time you read this, I'll have a software update applied and the issue will be resolved.
Read the rest of Visual Design for Safety
I was talking with a friend recently about some of their co-workers. In this case, they were complaining that another person couldn't do some tasks that my friend considered relatively easy things. Deal with an unusual restore situation, write some PowerShell to reconfigure servers, build somewhat complex (to them) queries, etc. Things that my friend thinks most database people should be able to do after a year of experience.
However, my friend questioned whether they were evaluating co-workers appropriately. How can one tell? I might be able to tell if a coworker is better or worse at a task, but what's my frame of reference and my experience. I might think them not capable at things I do well. At the same time, I can be wowed by someone with more experience with a technology I don't know well, such as Azure Data Factory, but not really be able to judge if they're average or an expert.
Read the rest of Am I Good At My Job?
Git has become a fantastic tool for me, and many other technologists, over the last ten years. It's almost ubiquitous in most of my clients, and so many people are comfortable with it. Many others aren't, which is why I started a Git series for DBAs (and other Ops people) on my blog.
Quite a few people asked me why I recommend git over a file share for storing code that a team of Ops people or DBAs might use. Why isn't a global file share a better choice in an organization? I think I have a few good reasons, but if you disagree, let me know in the discussion for this piece.
Read the rest of This is Why You Use Git for Scripts
I've always valued having a team of people I know and can work with. While I haven't had a lot of long-term jobs before Redgate, I have had a few positions that lasted more than a year and appreciated working with the same group for a long time. We might gain or lose a person, but overall, the structure of the team was the same day after day. This was a comfortable atmosphere, and I liked knowing who I was working with each day.
At Redgate, we have had some stable teams of people, but in our engineering area, we move teams around. There is an annual re-teaming each December/January where engineers can choose to leave their team and ask to transfer to another one. They get to put in their top three choices (or remain on their team), and we do a good job of trying to match up everyone's preferences. The number and charter of teams do change a bit each year, so engineers get visibility into the structure we're planning before they mark a preference. It seems from our internal reports that we match up 99% of engineers with their first or second choices (first choice is in the high 80s).
Read the rest of Un-Stable Teams
In the Phoenix Project (worth a read), there is a character called Brent, who is to go-to person for everything in IT. I don't know if this character was modeled after Brent Ozar, but I always picture him when I re-read the book, and I suspect he was that person in previous positions. I've been that person as well, and it's both exciting, fulfilling, and very stressful. At Redgate, that person has been Robert C, who is my go-to person for many questions.
In the DBA world, I think of Jeff Moden. He's been a prolific and incredible author over the years on many things SQL-related and is a huge proponent of others learning to write better code and better utilize the database platform more efficiently. I suspect in his company, he is the go-to person for most database-related questions and problems. I also suspect he solves most of them very well and has the influence (or power) to effect change.
Read the rest of Do You Have a Jeff?
Ransomware has been a growing and shrinking problem in the modern world. Every time I think that some new defenses and protections are preventing ransomware from being a problem, I see another issue. Recently, I saw Subway got hit with with an attack and a few friends have recently noted their companies were restoring systems after a portion of their network was locked down.
With the advent of Rasnsomware-as-a-service, where criminals deploy software and then sell access to others, better detection and protection become more important. As with any software, criminal human operators will use the ransomware software in different ways. That means that we don't necessarily have a simple threat that can be easily programmed against with anti-virus technology.
Read the rest of Ransomware vs. AI
Recently Robert Sterbal pointed out a podcast to me. This link is for Apple Podcasts, but it's for the Journal, which is on other platforms (I listened on Spotify). It's the story of a computer glitch in UK post office software, which resulted in quite a few local postmasters being criminally prosecuted, many convicted, and even a few committing suicide. It's a sad story, and it's complex, but there are some technology-related elements.
First, the overall story is Fujitsu sold the UK a point-of-sale system for post offices. There was a computer glitch here, which incorrectly calculated lots of totals and showed postmasters owing more money than they should. They were upset, called support, got nowhere and many were liable for paying money they didn't owe. The UK postal management hid information about the widespread nature of the problem, while prosecuting many local postmasters. Fujitsu support didn't disclose to callers how others were experiencing this same issue. This also coincided with a (an unrelated) law that changed saying computer systems were presumed correct and anyone accused of a crime had to prove the computer was wrong.
Read the rest of Serious Software Glitches
As a hobby, I coach volleyball for kids. Each year I meet new kids and parents and talk about my philosophy. Part of that is that I teach skills, athletics, how to compete, and more. However, I do emphasize as well that I'm trying to build better people, not just athletes. I want these kids to be better prepared for life, for job interviews, for future teachers and coaches, and for better relationships.
While I'm not perfect, or maybe not even a great person, I do try to improve myself on a regular basis. Or at least learn to be a better person to others. This isn't because of any ideal or goal, but because life is more fun when I have more skills. Whether that's working with objects or interacting with people. The smoother things go, the more I enjoy the world.
Read the rest of Advice To Be a Better Person
Brent wrote about what a good DBA looks like and challenged people to write a testimonial for them. There are some great comments on the post, and some funny ones. It's worth a read when you need a break from other work. I especially chuckled at the picture Brent used. I think ours was better.
That got me thinking: what do I want in a team? Or maybe, who do I want in a team?
Read the rest of The Team You Want
The economy might be good or bad for you right now. Some of that depends on where you live, what your employment situation is like, what your habits dictate about how you live life, and more. No matter what your situation, likely there are people around you that complain about the world and others who think things are fine. There are likely more of the former than the latter, but that's because humans tend to complain out loud more than they praise.
When people think there is an economic downtown for themselves, they may be more likely to engage in malicious activities. While I don't think most data professionals will start to hack other systems, or even their own employer's systems, there is evidence to support the idea that some might be susceptible to recruitment by bad actors. This piece references some research and warns security groups to be wary.
Read the rest of Rogue Colleagues
I was lucky enough to attend the very first PASS Summit in 1999. It was a brand new event, and while not overly large, it was busy and crowded in the basement of a Chicago Loop hotel. I had the chance to meet Kalen Delaney there and ask her a question. That was the highlight of the trip, but I also learned a lot about SQL Server 6.5 and 7 there, spending all the time I could in sessions and hanging out after each one listening to the questions others asked the speakers.
Since then, I have been lucky enough to go to most of the Summits. The event has changed a bit, and I look forward to going to see friends each year and re-connect with them. I've also loved meeting new people, often those I've corresponded with online. Each event has been memorable and exhausting at the same time. It's a busy week, and I can sometimes feel overwhelmed. However, it's always felt like it was worth the trip.
Read the rest of Looking Forward to the PASS Data Community Summit 2023
When Power BI was released, I was working with customers to try and adopt DevOps for their database code. At that time, I thought the Microsoft developers had made an extremely poor decision with the PBIX format in that it wasn't well suited for version control. In 2011, when it was first released, we not only had decades of knowledge about version control and the power of DevOps, but we'd also been living with the extremely poor format of code in Integration Services packages (among other data tools), that were poorly suited for understanding in version control systems.
Power BI has grown, and with the Fabric announcements, there are integrations with git for your workspace. There is also a Power BI Developer mode that saves your project in a folder structure rather than one file, which is better. I'm not sure it's great, especially if it doesn't enforce text in a way that ensures a developer can easily see changes between settings and understand them. I can certainly diff the XML in two SSIS packages, but that doesn't mean that it's easy to understand what has changed.
Read the rest of Power BI and Git
I was checking some arguments in the RESTORE command for SQL Server and saw that the MEDIAPASSWORD option was deprecated and marked as being removed at some point. That made sense, and I assumed that PASSWORD was the option to be used moving forward. However, that option is also marked as deprecated.
What should we do?
Read the rest of Removing Weak Security from SQL Server
Earlier this year I was watching someone present on DevOps and IaaC (Infrastructure as Code). The speaker was showing how they had worked with clients to implement tests and checks that evaluated whether their systems were deployed and the code was working as expected. They did this as the CI/QA environments weren't maintained, but rather spun up as needed when code was committed or tested.
What I found fascinating was the part in the talk where the speaker said they were "shifting right."
Read the rest of Shift Right
This was a provocative title: 6 ITOps Skills That Will Never Be Automated. In a time when AI use is growing quickly and many people fear for their jobs, it's nice to see someone writing about areas that AI will struggle to handle.
Ironically, lots of these articles are written by writers without much technical expertise and who are more likely than others to struggle in the age of AI. If a computer can mimic styles, less writers will be employed. That might be fitting with so many journalists hyping technologies without really understanding them, doing so in a way that creates stress among actual IT professionals.
Read the rest of The IT Jobs AI Can't Do
A group of people are protesting more cars in San Francisco by disabling self-driving cars with a cone on their hood. It is likely that the cone disables some sensors and won't let the cars move. For truly autonomous cars, this could be an issue if people could prevent them from working with small hacks. I could imagine a strip of duct tape or some other smaller obstacle that might also work and be less noticeable to anyone looking at the vehicle.
Ignoring the humor here, or the more-cars-in-cities debate, this is an interesting place where computing systems can fall down. People do find ways to hack sensors or other ways in which our software gets data from the real world, which can force behaviors we didn't expect. We might allow for these, or we might not. In this case, I don't know that the developers could do anything different, other than perhaps detect and alert the owners that the vehicle can't move until someone clears the obstacle. They could add more sensors, but I suspect hackers would just block those.
Read the rest of A Real World Clash with AIs
I've been very lucky in life to go to many conferences throughout my career. I've gone as both an attendee and a speaker, and I have found them to be valuable in helping me continue to grow and thrive in my career. This is across many employers, some of which I don't think had particularly great benefits. Plenty of people I worked with did not get to attend any conferences, and some of them felt resentful. I understand that, but I think that many employers are willing to invest in training and conferences when there is a return for them.
I started going to ComNet in DC, as a young professional wanting to learn. It was close enough to drive up and go for the free day and then return home. A few colleagues and I would make the trip once a year and talk with vendors, chat with attendees, and learn more about computer networking. Across a couple of years, we had moved from thinnet and thicknet to ethernet and IPX to IP, and this was a place where those technologies, and many more, were being discussed.
Read the rest of Taking Advantage of Conferences
A long time ago a software engineer advised me to try and ensure that I made my interfaces clear to users, especially those that are busy focusing on some other task the software enables. The phrase he used was to make a "Fisher-Price" interface, referring to the toy company that makes big and brightly colored buttons for their toys. The idea being that we didn't want to cram too many things on the screen or possibly confuse a user.
As a kid, I watched Mr. Roger's Neighborhood on television and enjoyed the show. Lots of kids had a similar view, but they might not have known that the main actor took care in choosing his words to convey concepts and ideas to children. In fact, the former producer said that the writers joked that his manner of speaking was its own language, called "Freddish."
Read the rest of Using Clear Language to Avoid Confusion
I was chatting with someone recently that's younger than me. They've got a good job and had some success learning, but they felt like their career wasn't progressing fast enough. This person still wanted to grow their career and their income. That's the place many people find themselves.
This individual was asking my advice since they thought I'd had some success and were looking for me to share some of the things that worked for me. I asked them this question: What have you done lately outside of work for your career?
Read the rest of What Have You Done Lately Outside of Work?
I chose the title slightly to poke at Stack Overflow (SO), but the same take expressed in this tweet could be said about SQL Server Central. It's not quite the same as anyone can answer questions on SQL Server Central.
The tweet is a (long) hot take from Jerry Nixon, a C# developer and MS evangelist in Denver. Essentially he says that a lot of the SO answers are wrong, especially as the software and languages change. Old answers are upvoted, and remain at the top of the list, even as newer answers might be better. People don't like the behavior on SO of moderators and people who post, which is something we've tried to avoid or limit here at SQL Server Central. We want there to be professional discussions. SO also doesn't allow much discussion or nuance in the questions or answers.
Read the rest of How Wrong is Stack Overflow?
Microsoft Fabric was announced at Build in May 2023. This is the next evolution of data warehousing from Microsoft, folding in Synapse and a number of other technologies to create a simpler location for storing and analyzing data. We've published some articles on the platform and there's a great presentation from Mr. Paul Andrew on Linked In. It's worth listening to, even in the background. Paul has a nice style and a great voice.
Part of this platform is OneLake. This is a data lake for your org, just one of them, and while it's able to store data in many formats, it's mainly optimized to read tabular data in the delta parquet format. This is essentially a compressed text file that allows for some transactional changes to the original data in parquet format.
Read the rest of Archive to the Lake
The great post-pandemic, post-Great-Resignation, hybrid/remote/in-the-office work debate continues. It seems almost every week I see more stories that report on, hype, attempt to prove, or otherwise stoke emotions about whether the future of work for many people is more likely in any of these three situations. One company mandates everyone back in the office, another goes completely remote, and it seems many can't decide and have some sort of hybrid approach.
Your boss might have strong opinions on one of these situations, which may or may not align with your organization. I see CEOs wanting one thing and individual managers sometimes wanting another. As much as I like remote working, and I've been based out of my home for over two decades, I also like seeing people and I am regularly traveling to the various Redgate offices (along with Grant and Ryan).
Read the rest of Flexible Work Policies are Preferred
I will admit that I don't know a lot about AI (Artificial Intelligence) systems and how they are built. I've been playing with them a bit and haven't been overly impressed with the results. I think some of this is that the my work is creative and I'm both used to being creative and I find the AIs less creative. And less accurate. And require a lot of editing. I don't mind editing, but not if it takes longer than just writing things myself.
From my understanding, a lot of the models behind AI systems (chatbots, recommenders, etc.) are built with humans giving them feedback on their responses in what's known as RLHF (Reinforcement Learning from Human Feedback). Essentially paid (often low paid) people that help to "guide" the AI into responses that are useful.
Read the rest of Constitutional AI
At Redgate Software, we've been trialing Copilot from GitHub with our developers. I managed to get access for this experiment and have tried a few things, though I'm not sure I've found it very useful. I'll continue to work with Copilot, but for now, I just don't find Copilot AI helping me with the types of tasks I do.
However, our chief of technology, Jeff Foster, was interviewed about how he sees AI and Copilot in Redgate. It's a good read, and I found a few fascinating things in the piece. First, I was worried about this a bit, as we have explicit guidelines in developers that say we can't take the output from an AI and put it in our codebase. We worry about legalities since we sell software. How does that work with Copilot?
Read the rest of We're Not Faster with AI
I heard a joke years ago that went something like this. When a developer gets a pull request for code review that's 100 lines long, they will open the file(s), look at the code, and ensure standards are being followed. They might run some code, they'd walk through the algorithm in their mind, and spend 10, 20, or more minutes examining how the change is built. If you give them a code review of 1000 lines, they'll just assume it's great and approve it in seconds.
I know that many developers don't find that funny. Often I meet people that think they're professional and they spend the time needed to examine the changes and ensure they work. I'm sure that many people do this often, and some people are very meticulous about their evaluation of the change. However, everyone gets busy and I know that often depending on who submitted the PR and how busy the reviewers are, the joke does reflect reality. The longer the PR, the less likely someone will either a) spend the time to carefully review it, or b) catch small mistakes.
Read the rest of Large PRs are Bad
I was working with a customer recently that has a development process that both made me cringe and struck me as very creative. In this case, the customer has software they have written when spawning a few databases for each new project that is created. There are three types of databases created for each project, with unique names for the project. The DDL and DML to create these initial databases are stored in a central database as a set of rows in a table.
To make changes to their software, developers create a project (with new databases) and then alter the application and the database to meet the requirements. These changes then need to be captured and applied to a development template database. There is a template for each of the three different databases created for projects. Changes made to these are processed and then added to the central database for new projects, and to upgrade existing ones.
Read the rest of Creative Development
Most of the code I've written for various employers was something that lived inside the organization. This was for internal use and no one outside of the organization ever used it. It was almost always code I'd written, so I wasn't ever worried about the provenance of the code (where it came from).
However, I'm sure I've copied code from somewhere online and used that in some way. I might have taken this function from Jeff Moden and dropped it on a production server where it was used to process a string. I'm sure I've taken code from various places and used it in administrative functions, like Ginger's code to manage a log file.
Read the rest of A Paste Policy
One trend for many organizations is to be data-driven. This means using data to make decisions at all levels, or at least support those decisions. This was popularized by many companies, and there is research to back up the claims that those organizations who do this have more success, more revenue, more profitability, etc. Much like DevOps, however, it seems many organizations don't want to actually invest in making data-driven decisions.
Instead, many opt to make decisions by HIPPO. If you've never seen this acronym, it stands for highest paid person's opinion. Many managers will defer to whoever is above them in the org chart, often someone that is paid more money than them. Even when there is data to make a different decision, the HIPPO is still used.
Read the rest of HIPPO vs. Data
I live on a working horse ranch. My wife boards, trains, and trims horses and has employees. Occasionally I have to help out with chores, or more often, fixing things. I was talking with a coworker recently and showing some pictures. He was asking why I do things like tap a bolt instead of hiring someone to do it.
I caught this excerpt from a book on cooking, which got me thinking. I don't love the flow here, as it wanders between cooking, economics, politics, social justice, and more. However, the core ideas of specialization and taking on a task was what I got out of the piece.
Read the rest of The Value of Doing Something
At the first job I had as a DBA, I had to build a new server. This was in the days of SQL Server 4.2, and I was combination DBA, sysadmin, and general help desk at a small company. With a software developer consultant, we ran some tests on various machines and then ordered a collection of parts from Compaq. Back in this time, they would only customer parts of the server. We unpacked our boxes with the server, extra drive bay, various SCSI drives, and extra RAM. We assembled and tested the machine and eventually put it into production.
In the years since then, I got out of the hardware business and left that to others. For awhile I worked in organizations with IT staff dedicated to building machines, but at some point we stopped doing that. The growth of VMWare and other hypervisors changed the paradigm for most organizations. For more than a decade, all the servers I've connected to are virtual machines running on hardware that my employer or a cloud provider owns and manages as a node in a cluster.
Read the rest of Bare Metal
Are you ready to get a new job? I'm not talking about you deciding you want a new job and working towards that goal, but rather getting surprised by a layoff or termination today. If your boss called you into his/her office in the next 10 minutes and told you that you were being terminated today, are you ready for that? Kendra Little wrote a nice post recently asking who might be prepared.
Most of us aren't. In fact, in my branding presentation, one of my slides has a screenshot of a post from LinkedIn. A friend noted that they walked into work and found out his company had been purchased. It wasn't a good day because they were also notified they were being laid off, hence the post on LinkedIn letting others know of their need to find one.
Read the rest of Planning for a Bad Day
I saw a great question on Twitter from Frank Pachot, a developer advocate of Yugabyte. He wrote: Without thinking how your preferred database deals with it, what do you expect if:
- session 1 starts to reads table T
- session 2 drops table T
- session 1 continues to read
The choices in his poll were: session 2 waits, session 2 fails, session 1 fails, both fail. My first thought was SQL Server and the default need for session 2 to get an exclusive lock. In that case, session 2 would wait. Most people answered that same way, but then Frank posted a follow-up with a link to his blog. The answer for Yugabyte is that session 1 fails as it gets the message that the table was deleted.
Read the rest of Concurrency Challenges Around Schema Changes
The world of work has been changing quite a bit over the last few years. The pandemic appeared to be a boon to technology workers, both by allowing us to work remotely, but also taking advantage of our skills to deploy more technology throughout the world. At the same time, late 2022 and 2023 have seen many companies pull back and cut their staff, including technical staff. It seems that in mid-2023, we have both a demand for technology workers and a lot of people looking for technology jobs.
A recent 2022 report on working in technology has some interesting insights. It is summarized at ZDNet, but you can see the original report here. The highlights from the report are that quite a few top performers left their jobs, recruitment doesn't work well, and finding top talent is hard. These aren't that surprising to many of us that actually write code or manage systems, but it is nice to see that founders and executives are reporting this (they were the ones surveyed in the report).
Read the rest of Highly Skilled
SQL Server databases have had a compatibility level for a long time. This is a setting that enables the database to process code as if it were a particular version. The levels go from 80 (SQL Server 2000) to 160 (SQL Server 2022). Each time (almost) a new version of SQL Server has been released, there has been a new compatibility level.
However, not all versions can support all compatibility levels. For example, my SQL 2022 instance can support levels back to 100 (SQL Server 2008). If I wanted to get a database to act like a version older than 2008, for example, 2005, I would need to install a SQL Server 2012 or older instance. There is a table of engine versions and supported compatibility levels on the ALTER DATABASE Docs page.
Read the rest of Actively Choosing Compatibility
There are a lot of technology people looking for jobs these days, especially after all the layoffs that have occurred in 2023. At the same time, I have a number of friends and clients that are struggling to hire qualified people. They get lots of candidates, but they are dismayed at how little the candidates they interview seem to know, or at least how little candidates know in the areas that their organization has needs.
Today, I want to help people get better at growing and managing their careers. Therefore I'm asking those of you that are employed to describe your job. Give us your title in the comments and then list 5-10 things that you've done lately. What types of queries have you written? What tasks did you complete with SQL Server or Azure SQL? Have you needed to research something to solve a problem? What knowledge or concepts helped you complete a task?
Read the rest of What's Your Job?
I saw a great post from DCAC on disaster plans and using them after a fire in an LA data center. DCAC wasn't affected, and I wouldn't expect them to be. Denny, Joey, Monica, John, Kerry, and Meagan are all experts in running systems efficiently and effectively. They think about how to ensure that things keep working in the event of disasters and have delivered presentations all over the world helping you become better at managing your own servers.
However, Denny brings up a great question in the post. How many days could you survive without IT systems? Days? Weeks? Have you ever been in that situation? I'm sure many of us have experienced a failure of some sort. An application crash, hardware dying, or most commonly, Internet access being cut. All of these are small disasters, which typically are fixed quickly. Hopefully, these aren't issues you experience every week. If you do, maybe you need to call DCAC or someone else and fix more fundamental issues.
Read the rest of How Many Days Can You Survive?
I have learned to really appreciate and enjoy Slack as a messaging tool. It's something I use daily, and a place where many inside my company communicate about all sorts of issues. There certainly can be an overload of channels, but for me, I add and prune channels regularly and it's a good way to segregate conversations.
I didn't feel that way when I started. At first, I resisted using it. Now I couldn't imagine not having it, but not everyone feels the same way. There's an article about a SaaS provider moving their company off Slack and instead using the Basecamp project management platform and keeping communications inside there.
Read the rest of Better Communication Through Less Messaging
Monitoring production databases is important when it's the systems are in production. Operations departments know that catching issues early, being proactive, and having data to troubleshoot issues make their job easier. Not having these things makes their job much more stressful.
Most of us work with data in some way and the availability of that is important. Certainly, security, integrity, and performance matter as well, but availability is key. Many organizations don't have any monitoring systems set up. Instead, they troubleshoot problems when someone files a ticket or calls. I'm amazed at this, though I know building and managing a monitoring system is hard and purchasing third-party products can be outside of your budget. Still, having something in place makes everyone's job easier.
Read the rest of Scaling Up Monitoring
I would guess that the majority of instances I've had to manage in my career were those that I didn't initially install and configure. I've inherited more instances than I would bother to count, and I often need to double-check what's been done in the past. As noted in the series on new jobs from Tracy and Josephine, there are a lot of settings to check and adjust to meet your standards. While backups are often my first priority, security is second. I usually want to know who the sysadmins are and ensure systems are patched and configured to reduce the attack surface area. There is one other security check that I think I haven't always been overly concerned about checking: password expiration. Read the rest of Validating Password Expiration
If you haven't seen the Build 2023 keynote, it's, well, interesting. At a surface level, it's focused on AI and delivers some demos that many of us might find to be useful and intriguing. I didn't attend the event (or watch it live), but I did see it a bit later and I made some notes, pausing the 30-minute talk a few times to think about what I'd seen. The opening lightly glosses over some of the AI enhancements to development tools and the environments that can be created quickly in GitHub or Azure. Some of us will like those, and maybe they'll grow on me, but I tend to prefer a development environment on my own hardware, where I have unlimited compute power at a fixed cost. The first big announcement is then showing Copilot technology, essentially some ChatGPT-like abilities, embedded into Windows 11. The demo shows asking Windows where settings are, with the response including buttons to take actions, like setting dark mode. Minorly useful, though I think Windows search works fine. I can type "env" and get the "edit environment variables" in the results. I still have to click through to change things, but this doesn't seem like a better use of AI, especially if I need to type "set dark mode" instead of "dark". Read the rest of AI Thoughts on the Build Keynote
Conveying information is a bit of an art and science. Many of us have written reports, graphs, charts, etc. at some point in our career. We've likely created some good ones and some bad ones that our clients love or hate. Perhaps if you're like me, you make a small attempt and then ask someone else to clean it up for you.
However, visualizations are important. In the modern world, many people want a visualization instead of a table of data, or at least alongside a table. That means we want to ensure we are conveying information well and not just picking the prettiest picture.
Read the rest of The Best Graph
Creating an API and using an API server of some sort has become a very common way of working with modern software, especially distributed systems. Whether developers move to microservices or something else, they often make more use of APIs than ever before. At Redgate, I've watched us work on new products, often beginning with a basic API at the command line and evolving from there.
However, does the database need an API? This article thinks so, though I don't love the piece. It seems to wander a bit amongst a few topics and doesn't always fit together. It's also trying to promote the Stargate API gateway for Cassandra. I don't know if this is a good idea, or if it really helps make developers more productive, but the idea of having an API for developers makes sense.
Read the rest of Database APIs
watashiato - n. curiosity about the impact you’ve had on the lives of the people you know, wondering which of your harmless actions or long-forgotten words might have altered the plot of their stories in ways you’ll never get to see.
I found this word in the Dictionary of Obscure Sorrows, which is excerpted at this site. This book includes new words that describe emotions in great depth. One of the descriptions was the word above, watashiato. Of the few dozen I read, this was the one that stood out to me, and it's something I've experienced. Maybe more than just being curious, though.
Read the rest of Watashiato
I was working with a client recently and one of the managers said they wanted to encourage innovation inside their organization. That sounded good, but as I thought about it, I wasn't sure what this means, or if it's really something that matters. Let's take a few examples and think about this in a modern organization that builds and operates software.
If I am a developer, I get a series of requirements or tickets where I have to write code. Perhaps I need to take inputs from a user, query a database, and produce a result. I might write code in C# 7 instead of C#6. Maybe I'd use a lambda instead of creating a new function. Perhaps I decide to ensure I use the stored procedure type with named parameters rather than submitting SQL as a batch. Is that innovative? It might be a better way to write code and improve the quality (or security or maintainability) of the code, but is it innovation?
Read the rest of Encouraging Innovation
We all experience technical debt when building software, especially software that has been around for years. Most of us have likely encountered code that we are loathe to touch and modify. At least, we don't touch it twice. We might change things once, but when it doesn't work as expected, most of us put it back in it's previous state and move on to something else.
At the same time, it's hard to define what technical debt is or point it out to managers, who often can't grasp the concept of why technical debt matters to a development team. They can't understand the reasons why new code built on older code takes longer and longer to produce. It's a nebulous concept that can baffle both people new to software development and those with years of experience.
Read the rest of Technical Debt
Many technical people dream of starting a company and having it grow to an IPO. The history of computing is littered with the successes and failures of companies that attempted this. As a young man in high school, I saw the explosive growth and success of Apple, dreaming of working for them or duplicating their success. In the late 90s and 2000s, I tried a few startups before I founded SQL Server Central with my partners. I had success with that company, though no IPO :(.
I've seen many friends start companies, often consulting ones, though a few have built products. Building and running any enterprise is hard, and startups are no exception. There was a post on Hacker News looking for advice and business lessons from others, and I enjoyed reading the responses. I wouldn't read much deeper than the first level of responses, as many of the subsequent comments are less interesting.
Read the rest of Startup Advice
It's my fault.
That's what I think if there is a security incident with my employer that involves the database. It's almost my first thought when I hear about issues at other organizations, thinking a technical person is at fault. Since I've been a developer and administrator, and I know how complex systems are, I usually stop myself and try to learn more before I assign blame.
Read the rest of The Public Perception of Security Issues
I play guitar as a hobby. Not great, but I enjoy it and find it relaxing. It's a good break from my day periodically, killing a few minutes before a meeting. It's also a nice way to unwind at night. I find it better than playing games or my phone or streaming more Netflix shows. I enjoy those as well, but there is something different about music.
I tend to use acoustic guitars, meaning no electronics. However, I have had electric guitars in the past, and have even used pedals to alter sounds. When I saw this article about design lessons from guitar pedals, I was intrigued. It has 5 lessons from these devices, which are for other digital gear. However, I think they could apply to software as well.
Read the rest of Design Lessons for Software
The last year has seen a number of large tech companies lay off large numbers of staff. The list for 2023 includes large companies, like Google (12,000), Amazon (9,000 this time), Microsoft (10,000), and Meta (10,000 this time), but also small companies like Zoom (1,300), Rapid (115), and Roku (200). It's not just tech companies, however, as Disney (7,000), Gap (500), 3M (6,000), and David's Bridal (9,236) are letting people go. There are plenty of other companies who have let people go, which is interesting to me as the economy has grown in the US, though profits were down. It's hard to know whether these layoffs are really important for all these companies or whether these layoffs are management's decision to group their bad news with everyone else's and take advantage of the opportunity to shrink labor costs.
In any case, layoffs are sad and stressful. Certainly, the people being let go are traumatized and I don't want to minimize the impact to their lives, but this can be hard for the survivors as well. This isn't just a Silicon Valley situation, but one that affects many employees all over the world. Whenever there is a large staffing change in an organization, those that remain can be traumatized and unproductive. This is one reason that public companies must notify and disclose layoffs to investors.
Read the rest of The Survivors
Who should create documentation for software? In many companies, it's the developers. In fact, in Redgate, often our developers are tasked with updating articles for products on our documentation site. We do have a streamlined process that has developers can submitting changes in some format (markdown? ) and an automation process that automatically updates the site as part of a release.
However, we also have marketing people who are in charge of external articles and communications. They ensure that our product learning and university content is updated. They work with technical people to produce, edit, and publish content. These could be developers, advocates such as Grant or Ryan, or even partners. Our Friends of Redgate also help with content.
Read the rest of Creating Documentation
One of the challenges for both database developers and administrators is doing more, often with less. Many companies continue to grow their database estate, both in width with more platforms, and in depth with more instances of the platforms they have. Some companies will look to shrink their staff, especially when adopting a cloud platform, while others may add more databases, but not increase staffing to match the additional load.
In either case, what many have found over the years is that the cost of labor is high. Both for developers that write code against databases, and administrators that manage those platforms. While licensing can seem to be a large number, compared to the cost of labor, it isn't usually a significant number.
Read the rest of A Case for Document Storage
One of the most amazing benefits of working at Redgate Software is the ability to take a sabbatical every five years. One of our staff wrote about this recently, and I found myself reflecting back on mine, as well as thinking forward.
The article notes that many people either learn or travel during theirs. That was somewhat of my experience, where I spent my first one learning skills and volunteering at home. My one-year look back is still interesting to revisit today. Unfortunately, my flagpole base failed in strong winds (sad face) and broke the pole. It's still on my list to rebuild a new one. I still look back on my volunteer time with fondness and try to get back to Habitat every year.
Read the rest of A Third Break
Data modeling is something that we should all be doing when altering the schema in our databases. I'd like to think that most people spend time here, but I don't think that's the case. I think plenty of people think "I need to store a piece of data" and they pick a string or numeric datatype and start stuffing in values. If in doubt, just pick a string. It's why I think we have lots of dates stored in string columns because that was someone's first thought.
There was a post recently that talked about storing data in its highest form. It was interesting to me because these are the type of decisions I try to make when designing a table. What is the best form in which to store data? The authors talk about picking not only a type that easily converts, but the fields that make it easiest to work with the data in different ways.
Read the rest of Data Modeling Information
Google has launched a version of their AlloyDB that can be installed on-premises. AlloyDB is a PostgreSQL compatible cloud database, a full-managed PaaS service. However, they are giving away a free developer edition and a paid for commercial license that can be installed where a customer wants to run it. The new product is called AlloyDB Omni.
I like Google's approach here. Obviously Google would prefer lots of their customers would move to the cloud version hosted in GCP (Google Cloud Platform), but they know that's not realistic. Even for customers that want to move to GCP, the customers might need to keep some workloads on-premises for a period time, so Google is giving them an option to modernize their workload with a new datastore that is PostgreSQL compatible, but running on-premises. Presumably this will be compatible with the cloud version and if customers want to, they can just shift to AlloyDB.
Read the rest of Coming Out of the Cloud
This article has a great opening quote. It says: "We are drowning in information but starved for knowledge". It's from John Naisbitt, who wrote the book, Megatrends in 1988. I think this quote can be very apropo in organizations as we have data, we have plenty of reports deriving information, but sometimes we don't have a lot of knowledge, especially when there has been turnover in our staff.
We can train new people on many things, but not everything. The knowledge of the culture, of what others know, of the little strange bugs or behaviors that can't get fixed, the tribal knowledge accumulated by living in an environment. These are the learnings that can't easily be replaced, and until they are, often new employees are less productive.
Read the rest of The Loss of Knowledge
Companies often want more data to help them make decisions on how they run their business. There has been this quest to gather and analyze as much data as possible to increase the efficiency of their operations to help reduce costs or increase profits. This has led to the importance of data as an asset, and the need for more data professionals in many organizations.
That's good for many of us that work with data.
Read the rest of Big Data Downsides
No, Redgate Software isn't hiring for SQL Prompt. I'm sure quite a few of you depend on SQL Prompt and would like more engineers working on it. Maybe a few of you would find that an interesting piece of software to work on.
Rather, a Prompt engineer is someone that works with AI, trying to get a system to produce better results. I can't decide if this sounds like an interesting job that stretches your brain or the equivalent of a mediocre developer that just keeps copying something from Stack Overflow, hitting compile, getting an error, and repeating that cycle.
Read the rest of A Prompt Engineer
There was a Slack thread at Redgate recently where a developer was showing some code where they decided to use the "extra" column from the information_schema.columns view. They were making decisions on how to detect certain metadata about a column based on the data in this column. Apparently, the data in here is overloaded for different options that might be set on a table.
This caught my eye because I had no idea there was a column named "extra" in this view. I flipped over to SSMS and decided to check what was being stored in here. To my surprise, there was no "extra" column. As I dug in a little deeper in the thread, I realized the developer was talking about Information_schema.columns in a MySQL database.
Read the rest of Information Schema Strangeness
I travel quite a bit every year. Over 20 trips in 2022 and five trips in the first quarter of 2023. To make life easier, I have a few routines that I use to ensure that travel goes smoothly and I don't forget things. One of those routines is using a parking service near the airport.
This company used to have a fairly manual process, though it improved over the years. The pandemic forced them to move to more contactless service, which I appreciated. I could make a reservation online, get a QR code, and use that to both enter and exit the facility without interacting with anyone or handling money. A bit safer, but the big win for me was that this process was quicker for me move into and out of the lot.
Read the rest of Living With Broken Software
I give talks on branding and managing your career at various events. Often I am helping people to better market and sell themselves if find new opportunities. One area that I talk about for data professionals is the online profile. I think having one is important, especially as many of our contacts with potential employers are made digitally.
I ran across an article that focuses on LinkedIn, specifically the profile you create there. It has some thoughts on recommendations, accomplishments, certifications, and more. I like the thoughts, and I do think they help showcase who you are as a professional working in some field. It is worth reading through the article and adjusting your profile to include these items.
Read the rest of Building a Better Profile
For much of my career, I've run SQL Server Central. A large part of the popularity of the site is from the forums, where people can pose questions about their struggles with SQL Server and get answers from the community. There are also some off-topic forums, where people discuss various things outside of databases. In here, we have discussions a about life, sports, and more. While we do expect people to maintain an air of professionalism and respect others, we don't try to moderate content.
That's how much of the Internet has worked, with various sites allowing users to post content, but not having any responsibility for what has been posted. The liability for that lies with the person doing the posting, which creates a thorny issue when users post anonymously. Setting that aside, I've been a proponent of this, not believing that Facebook or LinkedIn, or SQL Server Central ought to be liable for what users write and post. I do thinks users bear that responsibility.
Read the rest of Being Responsible for Data
Many organizations have been trying to find better ways to build and deploy software for their customers. Whether they deal with the general public or internal customers, we know that delivering software that customers use can be a competitive advantage. That's the goal of DevOps. While most developers and management want to do this, they sometimes forget what the goal is. Instead, they want to continue to work in a similar manner themselves while giving lip service to actual change. They often do this while pushing others to somehow produce more and better software inside the same system. I see this over and over inside various companies. Read the rest of Metrics and Measures.
The last three years have felt like a bit of a time warp. I'll run into someone or reach out to a friend, thinking that I haven't talked to them in a year or two. Then I realize that it's been 4 or 5 years because the 2020-2022 time frame felt like a year or less. A long year, but still, less than the three years that passed.
Life is mostly back to normal for me, with travel, events, vacations, coaching, work, etc. all similar to the way things were in 2019. At the same time, I've changed and my life has changed. Some of this is aging, some is life moving on, but some is from the pandemic.
Read the rest of How Normal Is Your World?
Almost every time that I attend an event, I'll end up meeting someone that has had security issues at their company. I'm always surprised how many people have had ransomware or other security problem that didn't get well publicized. It's not like everyone has had one, but out of 100 people, it seems there is at least one issue.
Many of us work with third-party companies for either products or services. It's become standard to use other firms for specific things your organization needs. However, since that's the practice, many of those firms you partner with have their own partners. After all, they're using other companies for specialized work just like you are.
Read the rest of Has Everyone Had a Data Breach?
One of the very common expectations from many SQL developers involves transactions. Many developers (database or application developers) think they can open a transaction, do something, open an inner transaction (nested), and then commit or rollback the inner transaction separate from the outer one.
If you've worked with explicit transactions and experimented with this a bit, then you know that this doesn't work. Recently Brent Ozar wrote a post on this as he had a client think that committing the inner transaction would release locks. It doesn't.
Read the rest of Nested Transactions
Security has become better and better in many organizations. At the same time, hackers and malicious actors are doing a better and better job of finding new ways to attack systems. Some work to target specific individuals, often because of government or industrial espionage. Most of us aren't likely to deal with those issues, unless we work with (or are) someone that is very important in a particular situation.
Instead, many of us deal with wider spread attacks that look to exploit vulnerabilities in technology or humans at scale, hoping to find the weak links. Lots of people I know have dealt with viruses in the past that shut down systems, and more recently, had to rebuild systems crippled by ransomware. Despite their best efforts, this often means lots of extra unexpected work, combined with the stress of falling behind on our commitments. We have plenty of other work to do.
Read the rest of Stale Data Causes Security Issues
The short answer is no.
The longer answer is it depends, and perhaps to be more complete, ChatGPT likely will help us produce the simple, tedious queries with much less effort.
There has been a lot of news about AI and ChatGPT and how well it performs for a computer. That last phrase is important because while it's impressive, I don't know that any of us would consider ChatGPT to be in our list of the Top 5 Dream Dinner Guests. The tool isn't that impressive compared to most people, especially our friends.
Read the rest of Will ChatGPT Write Our Queries?
There's a common party question about which 5 people would you invite to a dinner party? Often this is amended to include living or dead people, and it's often interesting to hear people tell you who they'd invite and why.
Since most of the people reading this work in technology in some way, I was wondering who you would invite to a party that's related to technology. Living or dead, however tangential, is there a list you can come up with?
Read the rest of The Top 5 Technologists
In the past, many businesses hired employees whose role was deciding which prices to charge for their goods or services. At one point, organizations largely set prices based on their costs, though over time they tend to look at their competitors and set similar prices. If, however, management from multiple companies went into a room and determined prices, this would be price-fixing.
Price fixing is illegal. In many countries, we would not allow different companies to work together in a way that might reduce competition or take advantage of consumers. However, that might be a struggle in the future as we find companies using various services to help them manage their systems.
Read the rest of The Danger of Sharing Data
Many of us work inside an organization that has a process for building and deploying software. We may find our org doing this well, or we may feel our process is poor with lots of room for improvement.
A lot of the discussion around how to be better at building software in the last ten years has been around the philosophy of DevOps. This concept doesn't really prescribe how to build software, but give you goals to aim for. That means you still need to take the ideas of flow, feedback, and learning and decide how you implement them with your staff. What practices do you follow to ensure you can deploy quickly anytime your code is done? These can include ensuring you've tested code, getting feedback from customers, and more.
Read the rest of Building Recommended Software Practices
Basecamp (formerly 37 Signals) is quitting the cloud. One of the founders gives some reasons, and he had some detail in a tweet on what they've spent in the cloud the last few years. Over USD$3mm on various services, though their costs in search seem very high. I don't, and haven't, run as busy a business as they do, so I don't know if they've truly done a good job architecting things and setting up services. They say they have, though I'd expect everyone to say and think that.
However, I'll assume they are correct and they can't optimize things any more than they are currently. Their decision makes some sense, and I agree with it. I've been surprised at the growth of the cloud, both in size and how quickly people are moving to the cloud. I've also been saying for years that if you have a steady or known workload, the cloud is likely very expensive.
Read the rest of The Cost of the Cloud
I saw this tweet recently, where Richie Rump asked what has changed in T-SQL since the SQL Server 2012 version. A few people from Microsoft responded that there were changes in all versions, and while I think some versions have few changes, I decided to look.
SQL Server 2012 introduced the window functions with the OVER() clause to SQL Server. This was a huge change in that many aggregate queries were much easier to write without needing complex GROUP BY lists and subqueries or unions to join together different data. While I'm not an expert by any means, I find lots of queries for reporting easier to write with the window functions, and I've grown to enjoy using these in code.
Read the rest of The Growth of T-SQL
One of the challenges for me when working with customers is getting them to think about how to change their software process. Often they want to solve their problems, but no one wants to alter the way they work. Whether that's the protocol for capturing code or managing servers, it seems that changing the way we work is hard for many people. They want everyone else to change. Or they want a magic tool that solves problems without them changing the way they work.
Unfortunately, I'm not a magician.
Read the rest of Coaching the Digital Transformation
When I first started working in technology in the 90s, it was a time of outsourcing lots of work overseas. Many large companies followed the wave of manufacturing in the 70s and 80s by many companies, including lots of semi-conductor manufacturers. I watched as a number of jobs moved overseas, though fortunately not mine.
In the early 2000s, I worked at a company as a manager, where I was involved in some of the discussions about outsourcing a lot of our IT operations to another company. It was a scary time for my friends, as the move would have left a lot of them looking for work. The CIO tried to say that many would get jobs with the outsourcing supplier, but couldn't say how many.
Read the rest of Outsourcing Operations
The senior advantage means more to me all the time, though often I feel there are more disadvantages to being around longer than advantages. Getting older is hard, especially physically, and I struggle with that. Having more wisdom, more tolerance, and more patience, are good things, but I'm not sure I would consciously make that trade.
In a business, senior often equates to more experience and time working on systems. Sometimes it means more skill as well, but usually, the internal knowledge of how our systems work inside of our environment is more valuable, even though it can be hard to get management to realize that fact.
Read the rest of The Senior Advantage
I had never heard of a vector database. I assumed this was a specialist type of database used for a particular problem domain, like a streaming database or graph database. There is a need for specialized platforms in certain situations, but I wasn't sure what a vector was. The description I saw for a vector database was that they "... are specifically designed to work with the unique characteristics of vector embeddings. They index data in a way that makes it easy to search and retrieve objects according to their numerical values."
That sounds like any database. However, I saw a few more articles on the hype and then some details about the ways in which this type of database is helpful. Essentially, this is a database designed to store the outputs from various Artificial Intelligence (AI) and Machine Learning (ML) models that examine unstructured data. Things like images, video, audio, and even text are turned into numerical values, or vectors. The vector database is designed to help index and then search these vectors.
Read the rest of The Rise of Vector Databases
Monitoring your SQL Server instances is important to ensure you can meet your SLAs. Availability, performance, reliability, quality, whatever you care about, it's important that whoever is responsible is looking at how the database is performing. At Redgate, we have multiple teams working on SQL Monitor to enhance and grow it to meet your needs.
A short while ago there was an internal conversation recently about page life expectancy. We've had some customers ask about this and setting alerts to watch this value. Our developers and sales engineers asked for a few thoughts from Grant and others on how to respond. There are a variety of opinions, some saying monitor it, some saying don't bother.
Read the rest of The Complexity of Metrics
The #SQLFamily is amazing, at least I think it is. Like many families, it's welcoming, supportive, and comforting. It's also maddening, frustrating, and exacerbating at times. Like most families, or at least the ones I know, it's not perfect, but it's what we have and at the end of the day, most of us get along with each other.
It's also an open group of people. In general, we welcome people with open arms and smiles. Those of us that are more visible or prominent are willing to listen to, help, and support anyone. I was overjoyed during the recent PASS Data Community Summit, where I had the chance to see so many people that I haven't seen in person in 2-3 years. I met many other interesting people for the first time and enjoyed the experience.
Read the rest of Speakers Are Normal People
I saw this article a few months ago, which talks about engineers at Facebook not knowing where their customers' personal data is stored. The engineers were being questioned in a legal matter, where they were asked to definitively state where all personal PII data for any human was stored by Facebook. Their answer was that they didn't think anyone in the company would be able to answer that question.
Facebook has been controversial over the years, and plenty of people dislike the way the company conducts business. I noticed no shortage of data people (and many others) commenting on this situation, saying that Facebook should be shut down because they don't know where data is being stored.
Read the rest of Losing Track of Data
We collect a lot of data in our databases. Not as much in bytes as a lot of the video/audio/TikTok/Instagram sites, but still enough that many of us are constantly adding storage to our systems. All this data is not only a challenge to manage, but it also means that we are regularly dealing with query tuning issues. Better code, indexes, and more become regular challenges with large volumes of data.
I am a big fan of trying to reduce the data you manage where possible. Archive, delete, remove older data, do something. This not only makes your systems easier to manage and improves performance, but it reduces your risk. Any PII data you have that might store is an ongoing risk in the event of a data breach. I don't pretend this is easy to do in any way, but it's a good idea.
Read the rest of The Challenge of Deleting Data
It couldn't have happened to a worse set of people, and I'm glad it did, but I'll still take a positive lesson out of this. A hacker sent a typo in a command to a botnet and lost control. That's kind of funny, and I'm glad it happened. The less botnets, the better, IMHO.
The coding in this software didn't have good error handling, which is a lesson in and of itself. Overall it seems many developers do a good job of error handling, but I still encounter more pieces of software that allow problematic input than I'd like. While we don't have great error handling in T-SQL, you can make some checks, and you should.
Read the rest of Minimize Typing
A while back I ran across a blog post that talks about the difference between database snapshots and database backups. There are certainly some similarities and some differences, as well as an overlap in the places where you might choose to use each of these technologies. Both might be useful as a way to recover from a bad code deployment, but both aren't necessarily helpful for a DR situation where the primary server has a catastrophic hardware failure.
It's often the case that we learn only a bit about some of the technologies in SQL Server. That's understandable as the platform has grown very complex, encompassing a vast array of technologies and options. There is often some overlap between them and possibly different places where you might choose to use one or the other and solve a problem.
Read the rest of Knowing Your Technologies
Many of you reading this probably work primarily in SQL. Even if you are a developer whose main language is something else, you write a lot of SQL. Even if you have an ORM writing the SQL that goes into production, I bet a lot of you are writing queries against a database to check that the data coming back in your application is correct.
As for me, I mostly work in SQL, with PowerShell and Python being second and third. I tried R for a while, but I think Python does everything R can do and it's much cleaner. I find R very cumbersome. I rarely write C# or experiment with anything else, but that's the nature of my job. PowerShell is important, as I do a bunch of DevOps and PoSh is a good choice to work with on the command line for gluing processes together.
Read the rest of The Programming Languages We Use
The pandemic forced many events online, which has worked fairly well inside many companies and certainly spurred some technological advances in how we meet and share information. With the pandemic ending or at least entering an endemic phase, there are many live events now, as well as some attempting to embrace a hybrid philosophy.
I went to two conferences in 2022 that embraced the hybrid concept: SQL Bits and the PASS Data Community Summit. I was just a speaker at the former, and mostly a speaker at the latter, though I saw quite a bit of the behind-the-scenes action from the organizers' perspective at the Data Community Summit.
Read the rest of Do Hybrid Conferences Work?
The US was hit with a number of storms over the Christmas holiday weekend. This disrupted air travel for many airlines and their customers, but one of the worst hit was Southwest Airlines. They accounted for most of the cancellations, over half of their scheduled flights at one point.
A number of places reported talking with Southwest employees who blamed the lack of tech investment by Southwest over time, noting this caught up with them. The Chief Operating Officer disagreed, saying that their scheduling system is the best in the world, even as the CEO noted that their scheduling software couldn't keep up and they fell back to manual operations.
Read the rest of Forgoing Tech Investments
There was an article on the worst technology of 2022 from the MIT Technology Review. I was hoping to get some sense of what was really useful or failure-prone technology, but it seems that the article delves into decisions on how to run technology or policy more than the actual technology.
The list is here, with my short comments on those that might apply as actual technology items:
Read the rest of The Best and Worst Tech of 2022
The world of technology has undergone an amazing revolution in my lifetime. Things that I thought were incredible in my youth have come and gone in the world. I'm sure that there were dramatic changes in technology across lots of other generations, but I wonder if they were as dramatic for others as they feel for me.
As I watch older movies, I'm amazed at the evolutions. I was watching an older sitcom recently and saw someone using a cordless phone. While we have these cordless mobile devices, there was a time when having a phone in my house that wasn't tethered to a spot on the wall was incredible. And I haven't seen one of these in years.
Read the rest of The Tech That is Gone
This is the last workday of 2022. Next week starts a new year, and as I've often done, I wanted to look back at the year. This time I decided to look back month by month, at some of the headlines and memorable data-related topics. I'm tackling things month-by-month.
In January there was a set of "tech experts" who shared their thoughts on the best database management systems. This one is worth a read for the humor involved. I wouldn't really consider many of these to be DBMSes. I thought about including this one as an April 1 joke, but it was a real story. I get asked for my opinion at times by writers researching a topic they don't understand. I hope I don't come across like a few of these people.
Read the rest of Looking Back at 2022
This month's T-SQL Tuesday invitation asked about end-of-year operations that you might take on your databases. I don't know how many of you follow any specific end-of-year routine, but you may.
However, that got me thinking about an end-of-year career evaluation. It's not something I've done, but I do often look back and forward and build some career goals. That has been helpful in ensuring I do some work on my career each year, even though I'm much closer to the end than the beginning.
Read the rest of The End of Year Career Evaluation
This weekend is Christmas, and likely many of you are not working hard today and will enjoy a long weekend until Tuesday. There are many other holidays at this time of year as well, and I hope you are enjoying the season with loved ones. I want to take a moment to wish everyone a Merry Christmas and Happy Holiday season.
As we come to the end of this year, it's a good time to take stock of life. Work slows down for many of us, and it's a time we think about family and friends. My wish is that you find yourself in a better position than last year. You are hopefully finding ways to cope and enjoy this world.
Read the rest of Merry Christmas and Happy Holidays 2022
I expected this to be an article from Glenn Berry. He often tries to convince me to upgrade. I think so he can justify buying his own new hardware.
In this case, however, this is a note from Tom's Hardware that it's a great time to upgrade your CPU. The article is a few weeks old, referencing Cyber Monday deals, and it notes that CPU sales are at a 30-year low. This means there's a lot of supply and retailers are trying to clear out stock.
Read the rest of CPU Upgrades
Ever since I heard about the SQLOS and all the work that went into producing an operating system for SQL Server inside of the host OS, I've thought that perhaps there might be value in specializing the environment for a database platform. Given the importance of databases, especially relational ones, and the need to get every bit of performance out of hardware, I wondered if we wouldn't see a version of SQL Server that is installed directly on the hardware, without a host OS.
In some sense, I guess that's what a PaaS database is in Azure, but I thought we might see that for the download-and-install version. I doubt that will ever happen now with the growth of the cloud.
Read the rest of The SPU
It's the holiday season and that means lots of promotions for various items to give as gifts. Black Friday took place recently and I had no shortage of advertisements that I saw for all sorts of things. From computers to televisions to headphones to any sort of tech gadget you can think of. Plenty of other types of gifts as well.
One item that kept cropping up for me was SSD drives. I saw an article in Tom's Hardware about prices crashing to all time lows. That dovetailed with my part of the Data Community Summit keynote where I remembered Jim Gray saying 1TB would soon be US$10,000 in 1999 at the first PASS conference. I looked back at a few orders I'd made and saw I paid $150 for 512GB in 2017 and $125 for 1TB in 2020. Now I could get quality portable, SATA, or even NVMe drives for under $100/1TB. Incredible.
Read the rest of Are You Still Using Portable Drives?
The decision to upgrade database servers can be a complicated one that involves features, costs, and support requirements. While I think many people would love to run database systems for ten years, often there are concerns about support, which effectively ends after five years for SQL Server. While you can purchase extended support, is that worth the cost? A hard question to answer, but one I've had to confront lately. When do you upgrade a database instance?
I've discussed this with a few customers, but it also came up in the context of SQL Server Central. We run a SQL Server 2016 database on Windows Server 2016. Someone recently sent a note that since 2016 is out of support, they'd like to upgrade the systems. They were thinking SQL Server 2019 on Windows Server 2022, but with the release of SQL Server 2022 that doesn't make sense.
Read the rest of Upgrading SQL Server Central
Time is the most valuable asset I have most days. Time is often a limiting factor that dictates my priorities at work. It also often determines my stress and enjoyment levels in life. Using time effectively can lower the former and increase the latter.
A few years ago I read Making Work Visible, which talks about work flow and how time can be wasted in teams. The book covers the 5 types of time thieves, and it helped me to better think about how I organize both my tasks at work and in my personal life. I think that has sometimes helped me find a way to get more done with less effort, and less stress.
Read the rest of The Time Thieves
Watching the evolution of SQL Server and the Azure SQL Database (ASD) variant has been interesting across the last decade. For a long time, ASD felt crippled compared to the on-premises product. The last few years, however, it seems that ASD is getting features first, which then slip into a release that I can download and run locally. It's a cloud-first database now, even though there isn't parity with both products.
In my career, I've worked with a number of platforms in production environments. One of those is Db2, though that's often felt like a legacy platform. I've rarely seen users starting new projects on Db2, though they might add some new databases, especially on AS/400, mainframe, or Linux platforms. I never thought Db2 was less capable than other choices, but I just don't see it used very often.
Read the rest of Cloud First Software
Years ago my son asked me to buy him a copy of The Unincorporated Man. It's a science fiction book about the future, economics, and sentient AI systems. It's the first part of a series of four, and I've enjoyed reading them all. I highly recommend them if you like science fiction.
In any case, I was reminded of the series when I saw this article from the New Yorker about selling shares in yourself. It's part of the premise behind the Unincorporated Man, where every person is their own corporation and sells shares of themselves to finance their careers. You could sell shares in your future earnings to finance education or some other investment in your future. In return, you have to pay back that value through dividends.
Read the rest of Investing in Another's Career
Artificial Intelligence (AI) systems continue to pervade many industries, usually where there is a lot of data and human developers struggle to build solutions that handle the complexities of the problem. Often the experts in these subject areas can't fully articulate the reasoning behind how and why they use data to arrive at some conclusion. We developers often struggle to get clear specifications from clients in simple cases, so I certainly understand why AI might be attractive in complex problem spaces.
I think AI is a promising way to try to tackle some of these issues, some of which are important to humans. These systems can achieve a focus and analysis of complex data in a way that very few people, if any, can. The sheer volume of data and myriad of relationships among the various different metrics captured eludes the ability of most people to properly analyze.
Read the rest of AI Medical Record Scanning
I saw a customer asking about Exasol recently, which is an in-memory, columnar database. I know nothing about it, and it might work great, but we don't support it. I didn't think much of it, as I'm sure the customer has a reason for choosing this platform.
Later, however, I wondered if this was a good idea. Another customer had inquired about Knack, which I had never heard of either. Not only that, it's an online database that appears to assemble its "tables" from data stored in other systems. Strange, but I'm sure it works well for some companies, especially those without many software developers.
Read the rest of The Challenge of New Platforms
I saw this posted on Twitter recently: "If you're always right, you're not learning. If you're never failing, you're not reaching. The objective is to be right. The objective is to succeed. But if you're always winning, you're undershooting your potential." –@JamesClear
I found that to be a very interesting view, especially as I think about moving my career, life, business, or anything else forward. I think about this in terms of the goals I've set for myself each year for work, or each season of coaching. In the past, I've sometimes tried to pick those items that I think I can accomplish.
Read the rest of Challenging Yourself
When I first started work as a software developer, I knew that getting an environment set up where I could compile a project might take a few hours or a few days. The complexities of how people built software projects, the dependencies, and more were handled in a very immature manner. These days I can often download a project from GitHub and compile it in minutes, with many of the issues handled automatically with various tools such as complex config files and package managers.
In the world of database development, I often find people struggle to get a database set up with a project. Whether you are coding in SQL, C#, or Java, many organizations haven't got a standard way of bringing up a database environment for developers. Some have a shared database for a team, which can be better, but can also lead to collisions of code as the pressure to get software built grows and team size increases.
Read the rest of Self-Service Databases
Quite a few of the bugs we see in production systems come from data that isn't handled well. Perhaps the developer never considered this data, or another bug lets data into a system that should never be recorded. These are often NULL values, but they could be other data values that are far out of the ordinary.
Where do we draw the line for edge cases? Is it anything that doesn't fit 95% of the data range? I see this number used in many fields, often manufacturing and other "physical endeavors". Is it the 80% rule, where we ensure 80% of data cases are covered, but 20% represent special handling?
Read the rest of What's the Edge Case?
When I was younger, it seemed that everyone I worked with in technology knew how to build a computer. Most knew how to work with a BIOS, were comfortable with command lines, and could assemble complex compiler directives into a Make file. Over time, it seems many people, especially Windows and MacOS users, became focused on the things they needed for their job, lacking a lot of knowledge about how computers process instructions and the low-level operations they perform.
There are plenty of very talented developers out there, and many great data modelers, but as I work with many of them that try to make the transition to DevOps, I see lots of uncertainty and tentative behavior. They often approach builds, automated tests, and deployments as though these are completely new skills they need to learn. New software that is foreign to them.
Read the rest of Losing Skills Because of Automation
One of the biggest challenges to becoming better at building, deploying, and operating software is the culture changes required. DevOps requires we work as a team, which can be hard to do. Often people have competing interests and goals. They don't trust each other as much as we'd like, and maybe more importantly, management doesn't trust the people doing the work.
There is plenty of blame thrown around when things don't work as expected (or aren't delivered). That's a poor culture in which to be creative or experimental. In reality, those things are a lot of what we do in software development. We aren't solving the same problem over and over, building a bridge or house that is mostly constructed as many others are. Instead, we are assembling more complex systems in different ways.
Read the rest of Psychological Safety
DevOps is a culture shift in your organization. Technology enables things inside of this culture, but the culture is important. If you don't know how to build a new culture and support individuals as they adopt it, often you're just renaming an existing broken process inside your organization.
Often, I think the failure to reach the efficient levels of a Spotify, Capital One, or any other DevOps success story is because of a failure of management to change culture.
Read the rest of DevOps is Culture
For years, those of us in technology have often worked outside of the core working hours for the rest of our organization. Whether this is being on-call, staying late, or coming in when asked. It's not uncommon for many developers and Ops staff to work 60, 70, or more hours to get things built/deployed/supported for our customers. Often we do this while juggling lots of tasks on a to-do list. While many things might be in a ticketing system, often I find there are tasks on a project plan, requests from team members, and other items which don't fit neatly into a ticket. I'm juggling more than one to-do list.
The line between work and personal life has often been blurred for me. I've taken work calls on the golf course, in an airport, late at night in my bathroom, while shopping with my wife, even at kid's sporting events. I regret not pushing back more to a few managers early in my career, but I also recognize that I (and many of you) are sometimes in poor cultural situations and we don't have a choice if we want to keep employed.
Read the rest of Life Organization
At this year's Data Community Summit 2022, I am a part of the day 2 keynote. I'm excited to deliver some thoughts and memories of the event. I'm mostly excited the Summit still exists, and I am looking forward to seeing many of you in Seattle in person again. If you haven't registered, think about it. This is the best place to meet other data professionals, network, and not only get knowledge from the experts, but ask them your question in person.
Someone brought to my attention the original event from PASS wasn't the Summit, but was the Compass '99 conference. I saw a recap of the event on the ITProToday site, which brought back memories. I had attended the event early in my career, convincing my boss that this data specific event would be good for both them and me. It was, and it led me to unfathomable success throughout my time working with databases, along with great personal joy.
Read the rest of From Compass '99 to Data Community Summit 2022
Today is Halloween, a holiday of costumes, candy, and scary movies. My family has enjoyed scary movies over the years, though not necessarily on this date. However, the theme has me thinking about the scary situations that you've encountered, or maybe those that worry you in your job.
I don't worry about too many scary things in my job. I do get some anxiety before some presentations, and there are times I've been nervous before recordings, but it's helpful for me to remember that none of these things are likely to be remembered for long if I do make a mistake. There's just so much content produced every week.
Read the rest of A Scary Situation
As I get older, I realize more and more the value of time. I also realize this is a very limited quantity for me, it is getting more precious each day, and it is one of the more valuable assets I have. Time with my wife, time with my kids and my extended family is precious. Time for hobbies, work, and more is difficult to come by. Well, not work. I have to work, but it gets in the way of things at times.
I've been learning to take time off from a busy life and enjoy the lack of trying to use that time. I find myself looking at email or checking some work item less and less on weekends and holidays. At least, I avoid working if I've had a productive week. Quite a bit of my work is independent and can be asynchronous with others. I might spend time during the week on something personal and then make-up that time on weekends. Often that's the case when I am called to be the emergency ranch worker if someone else isn't available.
Read the rest of Using Time Wisely
I find that many of us that work in technology want a solution to be great. Often perfect. Computers are so good at following instructions that we expect them to work as we intend all the time. We aim for being perfect in our code and we find “good enough” to be a low bar.
I think this is because many times something barely works and gets deployed because it’s good enough. Maybe the code can’t handle a load or corner cases and that’s caused us some sort of stress or pain. So we don’t want good enough. because we've been burned in the past, either by our code or someone else's.
Read the rest of Good Enough
A few years ago, one of my co-workers wanted to work remotely for the winter. They received permission and spent 3 or 4 months in France, waking up early to work for an hour or two, skiing in the late morning and early afternoon, and then coming back to work more at night.
I was quite jealous. Mostly because with family, this wasn't an option for me. Don't feel too bad for me as I get the chance to slide down mountains plenty during the winter.
Read the rest of Remote Remote Work
I saw a short post on meetings, which made me chuckle. The post said that meetings are work, and sometimes they get in the way of other work. I think that's a sentiment that many people have, especially when they are pressured to write code or manage systems and their daily calendar gets cluttered with meetings.
I find meetings to be very hit-and-miss for my job. Some are valuable and helpful, with plenty of others being unnecessary and an impediment to getting things done. It is hard to know when a meeting will be helpful or a waste of time, but often I find status meetings to be not worth the time required to attend. They very often are not worth the amount of attention they require, which has me wishing that I could just update status in a system or email rather than listening to others and presenting my own report.
Read the rest of The Work of Work
I got a taxi at Heathrow in the UK recently to go from one terminal to the other. I know, there are buses and trains, but it was late, I had my wife with a lot of luggage and I was worried the restaurant in the hotel would close before we managed another method of transport. As we were making our way over the driver noted there was a £5 charge to drop off at a terminal. That sounds silly, but I saw a billboard noting this and there is a link that this charge applies to almost all vehicles entering the airport terminal areas.
You might agree or disagree with the charge, or concern yourself with whether the airport authority or local government can (or should) add this as a little tax to those that drive to terminals. That isn't my focus today. Instead, I thought this was a somewhat amazing, DevOps experiment of how the digital world is remaking the real, analog one.
Read the rest of The Digital World Remaking the Analog One
I ran across an interview with Big Boi, a music musician and producer. It's an interesting look at someone that has had a lot of success and continues to work, mostly because he enjoys it. I'm sure the compensation matters somewhat, but that doesn't appear to be his driving force. Whether you like his music, it's an interesting look at someone who lives by his creativity.
As I read through the interview, a lot of the themes and quotes could be attributed to software developers. We constantly need to learn more and evolve how we write code if we want to be more successful. We have ideas, and often the code we write today isn't the way we'd write code to accomplish the same task last year. At least, I hope that we would learn, grow, and write new code.
Read the rest of Evolving Yourself
There have been a number of attacks in the last few years on source code. In fact, I saw a new one this week for an e-commerce Wordpress plugin. This time hackers got access to the distribution server for the company, Fishpig, and altered the plug-ins that their customers download.
A few years ago this was big news, with the SolarWinds exploit. There was also an attack on PyPy, a popular Python package that many people include in their code. There have been no shortages of problems in npm packages as well. I'm sure this has happened in other software packages, which is scary. In the days of DevOps where we publish code from a repository, an exploit against your developers might go unnoticed. Then again, maybe not.
Read the rest of Code Supply Chain Security
I do a little cloud work, but mostly I end up working with customers that are trying to build and migrate their systems to the cloud. The transition to cloud-first took place quicker than I expected, though certainly the pandemic of the last 3 years has continued to accelerate the moves by many organizations. While not all systems will move, for better or worse, many organizations consider the cloud for everything and then decide to accept or reject the notion for individual cases. That's the cloud-first world I see.
At a recent event, I had numerous people telling me that their groups were mandated to move many systems to the cloud in a lift-and-shift fashion, and many of the technologists weren't happy. That's a conversation for another day, but listening to them and others has taught me a few things. Some of these are experiences I've learned myself, and some are things from others, nuggets gained from their hard experiences.
Read the rest of Things I've Learned About the Cloud
Last month Brent Ozar posted that September was community tools month. He had a number of posts that highlighted tools and a number of other people (Francesco, Erik, VladDBA ) did as well. I didn't do any blogging because, well, it was a busy month for me. I was gone from home for 21 days, so a bit harried in life.
As a part of this effort, I saw Ken Fisher write about being a Script Magpie, which is an interesting analogy. A magpie is a bird that collects various shiny objects. Ken collects scripts in the same way. Things that catch his eye are added to his repo of scripts that he keeps handy.
Read the rest of Assembling Your Own Toolbox
There was a short but interesting post on the value of seniority. It's written from the perspective of someone that gets a new co-worker, but the co-worker is from the future. In fact, it's the future you. The post is actually titled senior engineers are living in the future.
We (hopefully) are more capable and impressive over time. I would hope most of us are learning more each day and our work reflects that. The code you write today should be better than what you wrote last month or last year. The choices you make, the bugs you solve, the experience you bring to work often does reflect your seniority. Not always, but often.
Read the rest of The Value of Seniority
When Microsoft started to talk about Intelligent Query Processing (IQP) before SQL Server 2017, I wasn't sure what to think. There was a diagram with 20 things on it, and only 5 were highlighted (you can see that in the article linked). That felt like something, but barely something. Over time, as we've gone through SQL Server 2019 and now 2022, the diagram now looks more filled in. In fact, now there are a lot of things filled in.
When I think about new features of SQL Server, I don't often spend a lot of time on the IQP stuff. First, I don't have to live with people yelling at me every day about slow performance. If I did, I might be more anxious to test and evaluate the changes. Second, these are mostly just hidden things that come with the product and (fingers crossed) improve my workload.
Read the rest of The Growth of IQP
Most of us are used to a database that lives on a server somewhere. It might be in our data center or a VM that exists somewhere, but it's really an on-premises type of infrastructure. Even if the VM is in AWS or Azure, this is a single system on a server that we control. We can add HA capabilities to this system, but the model is the same as if the database were on our development workstation.
Note: this doesn't matter if this is an RDBMS like SQL Server or PostgreSQL or a NoSQL type system, such as MongoDB or Neo4J.
Read the rest of Cloud Databases
Recently a friend started a new job that didn't have Slack used as a communication vehicle. The new employer was considering Teams, but most of the communication among employees was done through email or a live meeting.
At Redgate, we use Slack extensively. I also use Teams for some work with Microsoft, and in general, I find I prefer Slack. That might be because I don't spend enough time in Teams, but regardless, I find myself preferring email for discussions with MS employees over Teams. That's my preference, but really, what I value is the instantaneous, yet async, nature of using messaging tools over email.
Read the rest of The Communication Channels at Work
One of the challenges in moving to the cloud is achieving a similar level of performance to what you would get with dedicated hardware on-premises. The cloud is someone else's computer, and you are limited to the choices that the vendors allow you to rent. In many cases, you might even share the physical hardware resources with other customers, though many cloud providers have started to allow dedicated hardware reservations in the last few years.
One of the challenges of choosing a PaaS cloud database platform is that there are often restrictions that differ from a similar platform on-premises. This is often to ensure that one customer doesn't affect another, or that customers don't make changes that might prevent the vendor from supporting the system. Azure SQL Database gives us a database only, no instance features, though we do see a virtual master database.
Read the rest of Custom SQL Server on RDS
A big part of SQL Server Central has been members of the community answering questions for others. We have an amazing community and over the years, it's been helpful to so many people. I often get the thanks, but really the people at the top of the all-time list have been those that have helped many people. There are too many to list, but gilamonster, Jeff Moden, lynn-pettis, lowell, and more have been tremendous assets to our community.
Part of the inspiration for StackOverflow years ago was SQL Server Central and the popularity of people helping others. The main thing their founders didn't like from us (and other sites) was the need to register. Hence, the ability to be anonymous on Stack if you choose. No matter what format or style of Q&A you like, Stack and many other sites have found that there is a lot of desire to ask and answer questions online.
Read the rest of Paying It Forward
When I was getting my career going in Denver, Microsoft used to host lunch sessions every few weeks in their office. There were some Microsoft employees presenting and some community members. I learned a lot from both groups. At local user groups, more often than not, a community speaker was present, but there were plenty of people that worked for different vendor companies that delivered talks. I saw many of these people are knowledgeable in their area, and they taught me things I used in my job.
At some point, it started to seem that presenters that worked for companies weren't experts, and maybe more importantly, they weren't worth listening to because they were "selling something." In fact, a large community conference, and some small ones, banned "vendor" talks and the use of tools in presentations. Sometimes even free ones, if someone was promoting the software as the focus in their talk.
Read the rest of To Which Experts Should You Listen?
When we design software or databases, we have to make decisions right now. If you work in as many environments as I have, this means I'm often picking a tool, platform, version, technology, or something else without a complete set of information. In some cases, I'm actually writing code (SQL or C#/Java/etc.) without completely knowing the business problem I'm solving. I do know what someone described, but as many of us find out, those descriptions often prove to be lacking.
Over time, we refine the code with feedback from users, which is why DevOps has become so popular. We write the best code we can and then quickly alter code if we haven't hit the mark. We constantly adjust by making small changes and improvements.
Read the rest of The Challenge of Early Tech Decisions
I think that most of us feel like an impostor at some time in our lives. We will get asked to do something we've never done, with others expressing confidence in us. We may tackle the task, or we may not. We may succeed or we may not In any of those cases, we may still feel like an impostor, someone that isn't really qualified to do this thing. Many of us continue to feel this way in our careers, suffering from impostor's syndrome.
While I know that I'm good at my job, good at working with SQL Server and teaching others to do so, I still suffer from impostor syndrome at times. There are periods where my mind wonders if I've just gotten lucky and slipped through some evaluation process. Maybe my knowledge hasn't been well tested. Will someone like my boss, or their boss, question my skills at some point and get rid of me? Will I be able to find another job if that happens? Can I really compete with others out there? This isn't a constant or regular feeling, but I do experience it at times.
Read the rest of Fixing Impostor Syndrome
Developers, in general, are very optimistic about the code they write. This is likely one cause of their estimates of the time required being low, as well as the various bugs that slip through because of corner cases that appear for the problem being solved. Often developers think they've considered the various ways this code ought to work and covered all the possibilities. Usually we find they've not thought about the problem from other perspectives and need to adjust their code.
They often also feel that their code is superior to others, and that they can examine a problem in a new way. One of the reasons that I think many developers want to rewrite systems in some new technology or new way, embracing the Not-Invented-Here way of looking at other people's code. They want to write their own solution.
Read the rest of Developer Optimism
I just paid off my mobile phone. In my case, this is a phone from Google Fi, which provides fantastic service for me. The phone works well in any country I travel to, without any extra roaming costs outside the US for data. That works well, since it's a data device, and not really a voice device for me. Even overseas, lots of my calls are through some data service, like Facebook Messenger.
The decision for me now is whether to upgrade my phone. I've been getting some promotions and I've seen some deals from other retail outlets on phones, so I've been considering changing my Galaxy S20 for an S22 (perhaps plus) or a Pixel 6. There have been some improvements in cameras, and those are really the only change I'd notice.
Read the rest of When Is Technology Good Enough?
We've run Kubernetes inside Redgate for some research projects (like Spawn) and we are building some skills running this orchestrator. At the same time, we've had no shortage of challenges in keeping the clusters up at times, patching, fixing issues, upgrading to new configurations, etc. Like any software, there is work involved with managing the orchestrator.
I've watched Andrew Pruski and Anthony Nocentino write about containers and Kubernetes and overall they've made me view the clusters like email. It's useful and I want to use it, but I don't want to manage or administer it. I'd want some service like AKS or EKS instead. Let someone else build expertise.
Read the rest of How Hard is Kubernetes?
When I was starting my career, I expected to be a programmer. That's what people who wrote the software were called. At some point they adopted "developer" instead, shunning the programmer label. Now I see software engineers has replaced developer in many organizations. I'm not sure the job is much different than it was in 1990, other than the specific technologies used.
The DBA used to do a lot of system administration-type work on database instances. Check logs, set security, run backups, and maybe look at some queries. However, in many cases, their work was limited to things running inside the database software, or the database software itself (patches, related configuration for the host OS, etc.). I saw recently that DBAs have started to adopt the data (or database) engineer label as a new job title. Presumably, this pays more because, well, it sounds like it should. Data Professional sounds more comprehensive and skilled than Database Administrator. Database Engineer sounds better than both.
Read the rest of DBA to Data Engineer
Many of us know that reducing financial debt in our lives leads to a bit more security in our minds. We are better able to cope with unexpected expenses when we have the ability to get credit or pay for something in cash. If you've ever had a major house repair, car calamity, or (hopefully not) medical issue, you know what I mean. At least in the US. That last one is less of an issue in many other countries.
We know that our code has technical debt, and often we fight a battle with project managers and business people over whether we should spend time paying down that technical debt or developing new features. We certainly can do both, but often that's not a choice we are given. If we do get the chance to reduce technical debt, it's a tiny fraction of the time that is allocated to adding something new.
Read the rest of Reducing Debt and Increasing Security
I was giving a talk on DevOps recently and one of the questions from a person in the audience was how to get others to buy in. This person also had complaints that "DevOps" wouldn't work because their boss wasn't interested.
I think these are great examples of things that many developers and managers don't think about when they look to embrace DevOps. While there is no shortage of technical articles and videos on how to introduce automation or deliver features in a new way, what gets lost is the need for cultural change.
Read the rest of DevOps is Mostly About People
One of the sales managers at Redgate Software posted an origin story about how he came to work at Redgate Software. It's an interesting story, and while I don't work in sales or have the same origin story, I don't know that I could have seen the place I'm in now early in my career.
When I saw the video, it made me think of this movie scene, where Rafiki asks us to look beyond what we see. I think this is a hard thing for many of us, especially when many of us have been leaning on our skills in the past. What we know and what we are good at doing has led us to this place. Picking a new direction, or thinking in a new way might be hard.
Read the rest of Look Beyond What You See
I like the feedback system that Microsoft built for SQL Server. This used to be the Connect system, but all the bug reports and feature requests are now at feedback.azure.com. That's the place where you can send notes to Microsoft or vote on those that others have submitted.
In case you are wondering if this makes a difference, it does. Years ago, there pushback for Service Packs from Microsoft, and after a lot of lobbying from myself and others, and a number of votes, Microsoft released more Service Packs for a few versions. They've also made a few other changes in the product, or fixed bugs when enough people vote. Not always, but it does happen at times.
Read the rest of What Do We Want to See in SQL Server?
One of the more successful uses of AI (artificial intelligence) has been in the medical field. It seems that there is a tremendous amount of data, high variability in some aspects of the target of the data (the patient), and a need to constantly review very tedious volumes of data. A place where AI might, and has, made a difference to help humans, not replace them.
There's a neat article that dives into a bit of how a medical algorithm works for detecting sepsis. As you scroll, the article tries to explain in layman's terms what the AI system(s) are doing. It goes through the necessity of regular data, which I think is a good design to push humans to gather data. It then talks about a variety of neural networks being used to analyze data, based on previous training from humans and past results.
Read the rest of Digging into an AI Algorithm
I use Libsyn to store podcast files. They've been a reliable service for me for years. Once in awhile uploads are slow, but things seem to work. The other day I went to upload a file and got a status message that there was an issue with database maintenance. When I looked at the status page, I saw a few updates. This was for a platform issue, and I saw this set of updates. I have the timing with my own comment:
Read the rest of How Often Do You Update During Issues?
The last two years have changed the way many of us work in technology. The pandemic allowed many of us to work remotely, a challenge that many companies are struggling with. Or perhaps, management is struggling with and employees appreciate. In any case, no matter what your situation, the way we work has changed.
The change wasn't easy at times, and even though many people appreciated the extra time in their day from not commuting to work, they often had other challenges. Co-existing with others in a residence, finding a place to work that allows concentration, and even learning how to separate work from home life. These have been issues for many data professionals and other technologists. Even if we've had an easy time adapting, not all of our colleagues have felt the same way.
Read the rest of Being Reasonable
Microsoft wants more people to move their database workloads to Azure. They are constantly adding new features, capabilities, even tools to help people move databases to some part of Azure. It's working well, as the latest quarterly report shows tremendous growth in Azure revenue. We don't know precisely how SQL Server or Azure SQL databases impact this, but I certainly see more and more customers moving relational workloads to the cloud. Some to Azure, some to other providers.
In an effort to entice people, Microsoft has a promotion that they were running in March, which was asking SQL Server 2012 customers to move to Azure and save with free extended security updates. They reiterated the promotion on July 12, when SQL Server 2012 went out of support. They calculate savings and promote benefits, which might be tangible and desirable to some of you. After all, not having a supported platform isn't an option for some people.
Read the rest of The Azure Carrot
I found this article to be an interesting look at how we might add ethics to AI systems in one area. As the article points out, "... today there is no broadly accepted AI ethics framework, or means to enforce it. Clearly, ethical AI is a broad topic ...".
Glad someone is thinking, or many people are, but sad that we aren't really moving in a direction that creates a better system for us humans to work under, or be bound by. To be fair, I do think this is a very difficult topic and hard for any large group of people to agree on what should be done.
Read the rest of AI Ethics
I wrote the other day about a culture of allowing mistakes. We know mistakes are going to happen, so we ought to accept them. Even stupid ones. I make them at times, my wife does, my kid does, so I have been working to not get upset and emotional about them. Instead, hold someone accountable, let them fix things, and try to do better in the future. It's what I'd want, and what I am trying to do with my family, friends, kids I coach, and others.
Many of us share knowledge and information. One of the reasons I love events like SQL Saturday is that others share their knowledge. I especially appreciate is when someone shares a solution and how or why they build it. Hearing their stories, the reasons for their decisions, and what works well is great. I learn in a similar way from blogs, which is why we syndicate blogs at SQL Server Central and include them in the newsletter.
Read the rest of Share Your Mistakes
Building a culture that promotes teamwork, efficiency, retention, and other positive attributes can be challenging. While many organizations want accountability and responsibility, it is easy for managers to tip over into a blame-and-chastise pattern. When someone makes a mistake and a deadline is missed or a client gets upset, how do you respond?
I saw an article that gives some examples from the cybersecurity world about how to build a culture that allows mistakes and forgives them. We all make mistakes, and we should strive to correct them while working to avoid repeating the same mistake. But we will make mistakes again in the future, so how do we deal with them?
Read the rest of A Culture That Allows Mistakes
The cloud is truly someone else's computer, and it's not limitless. From the perspective of the individual, it might seem like you could scale infinitely if you need more boxes, storage, or networking, but ultimately the cloud vendor (Azure, AWS, GCP, etc.) needs to have spare machines around. Many of them do, and that works well at the scales in which they operate.
To a point.
Read the rest of The Limited Cloud
In this article on flexible working plans, number four makes a good point. What work is suited for live, in-person interaction? Which aspects of your job benefit from seeing others face-to-face?
Today I'm asking data professionals to think about it. I know we can do all our jobs remotely, after all, many of us have been doing that for at least a couple of years, and some for far longer. I can't remember the last time I actually needed to look at a physical server I wrote code for? Probably when I self-hosted SQL Server Central back in 2002-2003 and went to reboot a machine or add a disk drive. Even then, those weren't DBA or developer jobs, but really Operations hardware tasks.
Read the rest of Which Work Is Suited for In-Person?
There are plenty of people changing jobs in this economy. In fact, I keep hearing that companies are struggling to find people, though many of them are holding off making offers and looking for qualified people. Unlike the early 90s when anyone with an MCSE got hired without too many questions, companies are being picky today about who they hire.
I saw recently that Brent Ozar wrote his monthly post asking about who is hiring in the database community. There are a number of comments already with job listings, and I'm hoping there are more that appear when this publishes. If you know of a job, post a comment. The wider you spread the news, the better chance you'll get a person that best fits the position.
Read the rest of Finding a New Job
I've worked with computers for a long time. I've helped support various systems and applications, both desktop and servers. One of the most common tricks that has served me well is to press to oh-en-oh-eff-eff switch twice.
In other words, reboot.
Read the rest of Rebooting for a Reason
An engineer at Google recently claimed that one of the AI chatbots might have become sentient. Great headlines, and whether true or not, this might bring some notoriety to the engineer.. It certainly did, and it also resulted in the engineer being suspended from his job. It's entirely possible this person might be fired. Perhaps I'm cynical, but I think the more talented he is, the more likely he keeps his job. Less talented, likely fired. This might not be fair, but I am a realist. The more value someone brings, the more tolerance for missteps.
I give a talk on branding, and one of the things I do before giving you practical tips is to remind you to be cautious. A brand can be a very positive asset, but it can be a detriment as well. One of my stories is about Mark Jen, who Google fired after he blogged a few things about his employment. He was highly recruited and worked at Google for only a few days.
Read the rest of Be Smart
One of the things I've seen in working with cloud based resources is that you get a lot of them in your account quickly. A database might end up with a server, an IP address, a network, security groups, and more. A few clicks of the mouse in the Azure Portal can create a new Resource Group that doesn't just have the one thing you need, but 3, 4, or more other items.
In addition to the quantity, there are also the problems of namespaces in cloud resources. Some of the items you provision are publicly named in a domain, such as database.windows.net. In these cases, the resources need to be unique among all cloud customers. Just like domain names, this means that you might have collisions with your favorite name. While I might like jones.com, there are a few other people that would likewise prefer this. The rest of us might have to choose jones2.com, stevejonesincolorado.com, or some other variation. In large organizations, you might end up with LA34532345454.database.windows.net for an Azure SQL Database.
Read the rest of The Useful Cloud Tags
I've had a number of jobs in my career. In many instances, I left on good terms, and I'd go back to the organization if there was a job that suited me. I'd like to think that many of these organizations would rehire me if I applied, as I did good work and had success. I've always been pleased when someone is disappointed at my resignation.
Intel had a large layoff in 2015/2016 where over 13,000 employees were dismissed. The CEO at the time implemented a no re-hire policy, meaning those who were laid off, or I'm guessing those that quit during that time, were not eligible for employment in the future.
Read the rest of Returning to an Employer
When I watched Star Trek as a kid, I was amazed by the technology. Talking to the computer, the touch screens, the handheld communicators. We have most of those devices now, without the space travel. Hopefully that will start to change with all the efforts being made by various organizations. One of the things that always bothered me was the chance for mistakes. A mis-spoken (or mis-heard) command to a computer that didn't verify things as a human might. The chance to hit the wrong part of the screen as the starship moved. It seemed as though soft buttons would have allowed more mistakes than hard ones. Certainly humans make mistakes with physical switches, but I think I make more mistakes trying to hit a part of the screen in my Tesla than using one of the (few) buttons or wheels to change something. Interestingly enough, my 23 year old decided on a slightly older car because it had more physical buttons and fewer soft ones. Read the rest of Accounting for Typos
Data storage has always been a concern for data professionals. Early on in my career, we dealt with large ESDI, IDE, and SCSI drives, all of which would fail unexpectedly in servers. Sometimes after a few years, sometimes after a few weeks. We learned to use RAID and tape backups to ensure that our data was recoverable.
In many places tape was the long term storage medium used. These days, I know many people have moved to secondary disk storage of some sort, often rotating data across a few disk types that give you recovery for days, weeks, or longer. I don't know how long term storage work in Azure or AWS, but I assume some sort of combination of technologies are in use. I also know I don't trust them completely to be readily available and recoverable after a few years.
Read the rest of Long Term Storage
I guess that I was a remote employee that needed to onboard at a new company at one point. When Redgate Software bought SQL Server Central, I lived in Colorado and the company was (mostly) in Cambridge, UK. However, I was hired mostly to continue the same work I did at SQL Server Central, so I didn't really on board with the company. I signed a contract, and set up payroll paperwork, and that was it. I used my same workstation and laptop, albeit with a new account, password, and VPN, but almost everything was the same for me.
Over the last two years, we've hired many people at Redgate, with most of them being brought onboard remotely. Plenty of them have never seen any of our offices, and quite a few didn't meet any other employee in person for months. Our blog posted a couple of stories from people we've hired in the last year about the remote onboarding process.
Read the rest of Onboarding Remote Employees
Many of us work with data in some way that helps a customer better understand data, use it to make a decision, or support a some conclusion. The way we present data (or help others present data) can impact how well a consumer understands the information represented by the data. There are people whose job it is to design visualizations and ensure the final product communicates well, but often I find a developer or DBA is the person that gets asked for help.
What do we recommend when looking at a set of data and deciding how to format it? A few posts I saw recently help with decisions (when should I use a map) or basic knowledge (what is a square chart). I've seen presentations that help me understand the impact of my decisions and how to better design my informative reports for more people. In fact, I hope more people produce content on how and why they have built reports. It's helpful for many of us that would like to do a better job.
Read the rest of Better Communication of Information
At Techorama, I saw a keynote from Derek Martin of Microsoft. The talk was called Pain, Grief, Perseverance, and Technology, and it's worth seeing if you can find it live or recorded. David talks about his growth in life from a young man to a Principal Program Manager at Microsoft, husband, and father of six or seven. All the while dealing with a mental illness and learning to ask for help and to manage his busy life.
It's a great talk, though one I only lightly relate to. My life has been mostly easy and wonderful. Not always, but mostly. My problems are often minor ones. As I write this, I had surgery this morning to repair a painful ankle I've been dealing with for a few years. Things went smoothly and I expect rehab to be the same.
Read the rest of My Incredible Journey With Your Help
One of the things that I try to advocate for is that more of you actively manage your careers and find ways to improve your skills. I want you to be more impressive and find an amazing job for you. Not my job, because I have the best job in the world, but I hope you find a job that you love as much as I love mine.
It does take effort, work, and focus. It's something else to manage in life. I certainly don't want you to only care about work. After all, we work to live, not live to work.
Read the rest of Becoming a Better DBA
I was talking with a friend recently about technology. This individual is a person focused on business intelligence, originally a developer, but now an architect and consultant. They have a fair number of clients and have worked with them to build solutions to assist in analysis and decision-making for all sorts of organizations. This person has primarily worked in the Microsoft stack but has embraced NoSQL, Hadoop, and other technologies. In many ways they view the world as I do, using what works well for a particular situation without prejudice. They want to be effective, using whatever technology may be best in the current situation.
In their career, this person has extensive SQL Server Analysis Services experience and has built many cubes over the years that clients access with any number of front-end tools. I would guess that cube design and construction have made this person a lot of money over the years.
Read the rest of Are Cubes Dead?
It's been a decade-plus of the Not-Only-SQL (NoSQL) movement where a large variety of specialized database platforms have been developed and sold. It seems that there are so many different platforms for data stores that you can find one for whatever specialized type of data you are working with. However, is that what people are doing to store data in their applications?
I saw this piece on the return to the general-purpose database, postulating that a lot of the NoSQL database platforms have added additional capabilities that make them less specialized and more generalized. I've seen some of this, just as many relational platforms have added features that compete with one of the NoSQL classes of databases. The NoSQL datastores might be adding SQL-like features because some of these platforms are too specialized, and the vendors have decided they need to cover a slightly wider set of use cases.
Read the rest of The General Database Platform
Hacker News is a forum for experienced developers. While I think plenty of beginners lurk there, the discussions seem to be dominated by those who have worked in the technology or software industries for years. I saw an interesting article that noted Hacker News doesn't seem to discuss certifications and backed up the claim with some search results that show few results for various terms.
I think that many experienced people working in technology are wary of certifications. With a little practical experience, one quickly realizes how little certifications actually do to prove you know how to work with a platform, or with code. Most companies that hire someone with a certification will need to allocate some time to training them how to work on their systems. To be fair, this is the case with a college degree as well.
Read the rest of The Future of Certifications
Redgate had a discussion recently among our developers about our interview process and questions. There has been a standard question asking candidates about 2D arrays, but as one developer pointed out, we don't use these in our code base. So, why do we ask candidates about this topic?
The developers came up with a different question, actually a series of questions that ask about a class and then how to test parts of this class. We mostly work in C# in a DevOps culture, so this seemed like a good idea. They proposed a scenario with a few questions and then asked current developers to solve the questions and give feedback on the language, structure, and difficulty of the problem.
Read the rest of Crazy Interview Questions
I fly regularly. While I only had one trip during 2020 in the most dangerous time of the COVID pandemic, I flew a record number of miles in 2019 and had 10 trips in 2021. As 2022 progresses, I find myself back in the air and on the road, visiting new and familiar places for work and pleasure. I both enjoy the familiarity of known adventures and the excitement of new places. Already this year I have trips in place to three new places I've never visited and I'm hoping to plan more.
I usually fly to LAX a couple of times a year. I visit Redgate employees in the area or travel to some event. Occasionally it's a waypoint on a journey to some other city. As a result, I've gotten to know the scenery on the way, and the view is one I never get tired of.
Read the rest of Traveling Observations
Google started Project Zero to study zero-day vulnerabilities in systems. They want to improve security and safety, at least from a software and hardware standpoint. Its mission is to make it more difficult to find and exploit issues in software. They published an update recently that showed vendors patching their software quicker, at least according to data from the last few years.
The metrics show a change in patching from an average of 80 days a few years ago to 52 days in 2021. There were also patches that missed the 90-day deadline, though only one missed the 90-day time and also a 14-day grace period. They are also trying to help with the understanding of how well vendors are adapting to new challenges. I hope this pressures and inspires smaller vendors and even individual organizations to take security more seriously.
Read the rest of Project Zero for Better Security
All of us get into challenging situations. This might be a coding problem that we struggle to solve. It might be an argument with a co-worker. It could be feedback from a customer on how we dealt with them or maybe just how our app works. Our responses to a challenge might determine how we view the situation, our future actions, or even just how we react to life in the next few hours.
When someone struggles or feels as though they have failed in a situation, they tend to react in one of two ways. They can view the experience as though it were a test and they didn't pass. This is quite common among many people, who might become sad, angry, or defensive. They might defend their approach or look for an excuse. They might seek to put the whole situation behind them and move on. This is a common reaction and one I've had often.
Read the rest of Testing or Learning
Every once in a while, I find a list of places posted on the Internet with a "how many have you visited?" question attached. This might be US states, countries, cities, or even attractions. I love traveling with my wife and having new experiences, and as we've gotten older, we value the experiences more than any physical things. It's something that I hope we've passed on to our children as well.
My wife and I have been very blessed and have been able to travel a bit in life, though not as much as we would have liked with three children and a ranch of horses. We made it a goal about 6 years ago to try and visit a new country every year and experience life in another culture. The pandemic limited our travel for a few years, but we are starting to make plans to travel a bit more and gather some data for our minds.
Read the rest of Gathering Data Through Life
I have upgraded lots of SQL Servers from one version to the next, and for the most part, the process has been smooth. That's not always the case, and there have been some long nights where the Operations staff had to scramble to fix things, script out old logins, call Microsoft support, and perform various data exports and imports to get a new instance running. I've been a part of quite a few of those teams.
Most of my upgrades were with relatively small databases, at least small for that time period. However, I have upgraded a few "large" databases in the past. We had a 400GB database on SQL Server 6.5 in 1999 that was a challenge to move to SQL Server 7. I also upgraded an 800GB database in 2001 from 6.5 to 2000 for our Financial team, which involved a lot of stress.
Read the rest of Migrating a Large Database
Every once in a while I find a fun post that makes fun of programming. I think because the author is struggling with some piece of code, they write about the craziness of software development, often with an analogy that compares building something in the real world to how software programs get written. Here's one, with a bridge construction analogy, that degenerates into a rant on the disparate ways in which different programmers work together and how the entire world of the Internet is held together in a constant patch-fix loop.
I do think that most large software applications have a decent amount of inconsistency. I'd suspect most large databases (in terms of object count), also have a variety of techniques, tools, and standards in use. That's because the tooling that we use to build things change. We don't have the constraints of the real world, which is why we could have a stored procedure using a subquery to calculate a running total and another one that uses the OVER() clause.
Read the rest of The Craziness of Code
A number of you likely use Atlassian products like Jira, Confluence, Opsgenie, or something else. You might have been affected by a large outage they had (post incident blog, Company Q&A, TechRepublic report) recently which lasted at least 9 days. I don't know if all customers have their data back and are working, but this was a surprisingly poorly handled incident according to a number of reports from customers. There's a great write-up from the outside that you might want to read.
The bottom line in this issue is that Atlassian looked to deactivate a legacy product with a script, but they apparently didn't communicate well among their teams. The script ended up using the wrong customer IDs and also marked the sites for permanent removal, not temporary removal (soft delete). While they supposedly test their restore capabilities, they weren't prepared for partial restores of subsites. I'm guessing this is likely a partial database restore, which many of us know is way more complex than a full database restore.
Read the rest of Scripting Makes Mistakes Easier Than Ever
Why would you use SSMS/T-SQL over PowerShell (PoSh)? When is T-SQL directly a better option than PoSh? That's a question I ask myself regularly as I see articles and blogs that discuss how to accomplish a particular task using one tool or the other. There is plenty of overlap in the capabilities for each language when it comes to working with SQL Server, so this is a decision I think about regularly. This is especially true if you use dbatools.
There also appears to be a bias towards one tool or the other for each individual. Many people traditionally have used T-SQL to accomplish most database tasks, and they tend to always look for a solution with a script in SSMS. Others are excited by PoSh and I have seen plenty of questions on the SQL Server Central forums asking how to structure their code in that language. In both cases, there is no shortage of people that argue that you should use T-SQL instead of PoSh or vice versa.
Read the rest of Using T-SQL over PoSh
It's 2022, two years into a pandemic, and it finally seems that many things are returning to the way they were in 2019. Life outside of work is fairly normal for me, and I have a number of trips. Work is starting to pick up and we've seen a few conferences and SQL Saturday events pop up on the schedule.
Work is still mostly remote for many technology workers. Not all, and some companies are starting to require workers to come back to the office, part or full time. My son got a job as a software engineer and he has to go to the office every day for six months. After that, if he proves himself reliable, he can work at home 2-3 days a week.
Read the rest of A Third Space
Many of us have spent time looking through manuals or the documentation for some software or product. I know I'm on the MS docs site regularly for work, and there is no shortage of times I've used various manuals to help me fix something around the house. We usually use a manual when we want to learn how something is supposed to work, or how to get it to do what we want.
I saw a post on a personal user manual that I thought was a good idea for some people, maybe many people. This isn't a manual for how you should live your life or work, but rather, how others might interact with you. This manual describes how you work, what motivates you, stimulates you, what pleases you, and even the environment in which are most productive.
Read the rest of Your User Manual
I'm not great at building charts and graphs. I can build a basic chart, but I often depend on the tooling I use to size, scale, etc. appropriately for whatever I'm graphing. That, or I just use a basic graph that starts from zero and has some sort of linear scale. Or I just present a table of numbers.
There are plenty of misleading charts, especially used by the media that want to show some particular aspect of data that suits the story they are reporting. Many of these misleading charts often don't start at zero, and they end up scaling in a way that can confuse people.
Read the rest of When to Use Zero
I like SQL Server as a database. I think it's very complete, solves most of my problems, and is easy to use in work. It costs money, but less than some others. It's also more complete to me than some of the open source databases out there. That being said, I think most of the top five or six relational platforms would work for me and I wouldn't hesitate to use them.
I ran across a post from Steinar Gunderson, who worked at Oracle on the MySQL team. It's on his last day there, and it's a bit of a why did he leave. I like that he notes he found a better opportunity, but he digs in deeper. Why did he look for a new opportunity?
Read the rest of Moving Away From MySQL
Software development is complex these days. Maybe it's always been complex, but the shorter timelines and greater vision of what is currently happening in development teams mean that we can see a lot is going on at any particular time. Really, once you get beyond a handful of developers, the coding changes look like a chaotic ballet that may (or may not) be in sync across developers. Modern work tracking tools make this easier, and for any particular developer, it isn't too hard to look at a Kanban board linked to a git repository and track the changes that have taken place.
There is always more work to tackle than can be done at the current moment, so it is important to add new tasks to a backlog. Building those, and tracking what is needed, is a bit of an art. What do you write down, how much detail, and what level of effort is needed. Paul Andrew wrote a good post on what makes a good backlog story, using an example from his work. He looks at the need for infrastructure in this case, but that would be a common request these days when DevOps and the cloud require some blurring of the line between code and the resources it runs on.
Read the rest of The Backlog
Early in my career, I worked at a few smaller companies where a problem with the computer systems meant I went into the office and stayed until things were working. As I went to a few larger companies, I wasn't alone when a system went down, but the process was mostly the same. We figured out what was wrong and found a way to fix or replace things, occasionally with help from a vendor. Those were the (not necessarily) good days before our internal networks were connected to a public Internet.
Companies developed formal incident response plans to deal with various issues, whether these were problems we caused ourselves or failures of an application. I had the fortune, or misfortune, to be involved in more than a few issues and learned a great deal in how to solve problems as well as how to manage the impact to a large number of employees.
Read the rest of Advanced Incident Response
Jupyter Notebooks are a popular way of consolidating a number of code batches together and them executing them as separate batches or all together. These are essentially a document with notes and code, all of which are stored together. Databricks can run these, as can a number of other services. They are also integrated into various tools, like Azure Data Studio.
Just like many other technologies, we'd like to execute these in a way that suits our environment. While running these in a tool like ADS works fine, we might want to schedule these and check the results later. You can run notebooks with PowerShell, but keeping around results and managing the data is hard. Since many of us use SQL Server or Azure SQL Database, wouldn't it be great if we could find a way to have these run inside SQL Server and keep the results around?
Read the rest of Executing Notebooks in a Stored Procedure
At SQL Bits a few weeks back there was a community keynote on Friday. Ben Weissman and Rob Sewell put this together as a fun way of involving others. They invited a number of people to each take 5 minutes and talk about their favorite part of the data platform, which encompasses a wide variety of technologies and products. There were a few people who chose Purview, Azure networking, and quite a few ways of using Power BI. Me? I choose the Create button in Azure. Really, Azure itself because of the amazing array of things that you can deploy in minutes. It's stunning to me to think back to the mid-2000s, when I first saw the Azure platform at TechEd and a Microsoft employee demo'd a key-value table, literally a two-column table. They were trying to sell this as a great way to deploy lookup data to mobile sales applications at scale. I was less than impressed. Read the rest of My Favorite Data Platform Feature
I love this idea from Ken Fisher: saving your work. I don't act as a DBA anymore, but when I did, I did something similar. We often logged the scripts we used in a file, as a part of a log, so that if we broke something and another DBA got a new ticket, they could check what you had done. Over the years, we tried two different methods. First was using the desktop of the instance itself, since we often went to a room to log into the server in those days😉
The second way was in an Exchange public folder, where we added a new entry for each day. This way we could note the server and the scripts run. Since most tickets were dated, we could easily find the scripts if we were looking at a ticket. Since a user often updated or re-opened the ticket, we could use the public folder as a central note location from the DBA team. We could even point to this folder for our ISO and SOX auditors to show them what had been logged by people who supported the systems.
Read the rest of Easily Repeating Work
Passwords aren't going anywhere. While I would have thought there would be more advances by this time, and there are, the basic password is still required in many places, especially for resetting an account. Even those MFA places where I can click a notification or enter a code, I still sometimes need a password.
That's fine, and I think MFA is a good solution, but it doesn't alleviate the need to have a strong password. Troy Hunt has written about this topic because we as a collective do a poor job of building passwords. Especially with regards to length. Some of that is poor app (and database) design where we unnecessarily limit password length. However, some of the issues are our fault, as we continue as a group to use poor habits and practices.
Read the rest of The Level of Security These Days
I'm a do-it-yourself (DIY) person in many ways. I work on equipment on the ranch, repair or improve things in the house, and often tackle electrical or mechanical tasks. I've replaced hydraulic hoses, changed oil, replaced brakes, changed outlets, wired up lights, and more. I keep a hand cleaning degreaser handy and use it regularly.
I think many people who work in software development are the same way, and often I find they want to rebuild things themselves rather than adopt or improve what's there. Many organizations have a fair amount of NIH Syndrome inside that results in money and resources spent on software that might be better used elsewhere. I think some of this is going away as current trends in software make use of APIs, OSS, and services to cobble together a workflow rather than just building something from scratch.
Read the rest of The Right to Repair
I don't know if I'd be a chief, but I am glad that more companies are recognizing that data is one of the most valuable assets in most companies and starting to hire Chief Data Officers (CDO) to manage governance and security projects, as well as oversee data quality and management. With Chief Information Officers (CIO) and Chief Technology Officers (CTO), it appears that technology is becoming a force in the executive ranks.
Most of the companies I've worked for only had one chief-anything to do with technology. It seemed that anything to do with computing fell under that individual. Adding a CIO and, now, a CDO shows that there there is a complexity in working with computer systems. The infrastructure and architectures of systems are important, but how we capitalize on information is crucial. I see a CIO as more of a business role than a technology one.
Read the rest of The Chief Data Officer
Years ago I set up an email account for my son using Gmail, periodically forwarding things I thought he might find interesting. One day I was with him at his PC and asked if he'd seen something from me. He said he hadn't and opened his mail. He had dozens of emails, many of them marketing. I asked him why he didn't delete some of the obvious marketing ones that he didn't care about. He pointed to the sidebar, where the usage of his account was listed. It showed a few percent of the 10GB he was assigned in use. He said if the usage got too high, he would. For now, he would rely on search to find things.
That was fascinating to me. I'd grown up with limited space, used folders for organization, and pruned out anything old or useless. It was an eye-opening conversation to me on the difference in how generations looked at computing. That same thing is happening on a larger scale. As this piece shows, newer generations are approaching the way they use computers in a completely new way. They don't even necessarily know how to find items on their computer by browsing, which is strange for older users. It's a trend that vendors embrace. In Windows, Microsoft surfaces "Documents", "Pictures", "Music", etc., and I find many people have no idea where those folders actually are in the file system.
Read the rest of The Challenges of Changing Software Tools
I ran across a piece on the fall of the DBA (or part of a piece), and wondered if this is the time when the DBA is disappearing. I do know that plenty of organizations never formally adopted the DBA as a position, and plenty more have abandoned it. Not that there aren't people managing databases, but they have moved into Database Engineer, SRE, or plain old sysadmin. They might even have another job but function as an accidental DBA.
The DBA role has often been a mix of many things. Certainly, this is a tactical, operations role that keeps systems running, dealing with performance, security, and availability. DBAs also fix quality issues in production, deploy code changes, and often advise developers on what works well and what doesn't. They tend to be a bit of insurance against things going wrong as well as the scapegoat when anything does go wrong. I found the DBA role to be a constantly changing, never-ending grab-bag of challenges on a regular basis.
Read the rest of Is the Time of the DBA Ending?
Last year a company was testing their software in a dev/test environment and accidentally sent out an "Integration email" to their customers. They blamed this on an intern, though a little jokingly, and plenty of people shared their own stories about making mistakes.
I did this once, and only once. I was doing some software testing on a copy of the production database and accidentally sent out thousands of emails to customers. I hadn't scrubbed the email addresses in the copy, but since I also hadn't scrubbed the data, customers got an "extra" email for their account and few complained. My boss, however, complained quite a bit and with good reason.
Read the rest of The Test Mistake
The world of Information Technology has changed a lot during my career, but in some ways, many things remain the same. There are still mainframes in use, and plenty of jobs. Lots of companies maintain their infrastructure in data centers, even though some are turning to private clouds for their infrastructure. There are plenty of jobs in that area as well.
There is a trend to consolidate resources into a bucket and then allocate those as needed for various applications and services. This is what mainframes do, albeit at a very high degree of reliability (and cost). The cloud seeks to replicate this, not with high reliability, but with redundancy, expecting systems to fail. In fact, plenty of the data centers these days involve a lot of similar hardware systems tied together with software, with failed systems being left in place rather than replaced.
Read the rest of Less is More
SQL Server is constantly growing and changing, as are most database platforms. There are lots of platform changes, among them enhancements to the T-SQL language. Microsoft has added window functions, in-memory structures, the ability to execute code in other languages, and more. Some of these features are well built and some need more work. What's always interesting to me is what actually gets built and what doesn't.
There was an article recently on evaluating features in an RDBMS, and the article uses the JSON data type in Google's Big Query as an example. The evaluation is interesting, examining whether the feature actually helps the user, or if it is mostly marketing. In this case, the feature is outside of the "normal" conventions of the platform, but it is useful.
Read the rest of The Usefulness of Database Features
A saw a tweet from Brent Ozar about USB ports on slot machines to charge your devices. There are also wireless charging pads. Convenient, but also potential security problems, especially for IT workers that hold privileged access to code or data.
Please, don't use public USB ports for charging a device. You never know if there is any data access taking place. Yes, I know that most phones ask you to approve things, but have you ever hit the wrong button on your UI? Know how to turn it off? How quickly would data move or malware install? Just don't plug in.
Read the rest of A Real World Security Reminder
I saw a question posted recently on what data is included in a full SQL Server database backup. I hadn't seen that question in some time, but the post was a good reminder that this is not an intuitive concept, and new data professionals might not understand how a full backup works. If you don't know, you should do a little research (and write a #SQLNewBlogger post for yourself).
The way a SQL Server backup works, either with an on-premises install or the Azure SQL Database version, is well known and documented. Even if you can't make a "normal full backup" in Azure SQL Database, the process is the same. You don't have to run the backup, as Azure does that for you, but you can specify a restore and understand which data will be available in your restored database.
Read the rest of Backup Architecture
I caught this link in a Brent Ozar newsletter. It's a post from Alice Goldfuss noting she's on the market for hiring. Not that she's looking for a job, but rather, saying she's willing to talk to you if you want to hire her. Quite a bold announcement, but she's quite a talent that many organizations might want. I don't know that many of us could post something and have companies come find you, but I did like the list of items she lays out for a hiring conversation.
These are the types of questions that you might keep handy, even as notes, for your next interview. I've often had a series of questions like this that I ask when I go into an interview for a job. I've learned that I need to be curious and careful if I want a successful position for both me and the company. Both of us are trying to put our best foot forward, sometimes stretching the positives and minimizing the downsides. That means that both sides can be surprised later by how the relationship actually proceeds.
Read the rest of An Active Job Search
I'm not sure this blog that seems to talk about the problems of NoSQL databases in general makes a lot of sense. If you read the comments, you certainly see lots of complaints, but I also think the post isn't well written. It lists the problems of RDBMSes as possibly deleting all tables while changing a key or being unable to add a column easily. While I don't know about all RDBMSes, I don't know any that could lose all data with anything less than DROP DATABASE.
At the same time, the NoSQL complaints and problems seem to be generally presented, which isn't good. The various NoSQL flavors of databases vary widely and the way you look at a columnar or graph database is much different than a document database. Really, the piece ought to be separated to look at a certain class of NoSQL database compared with RDBMSes.
Read the rest of The NoSQL Rise and Fall
Jeff Atwood has had quite a bit of success in his career, having been a founder of StackExchange and Discourse. He's a fellow alumnus of the University of Virginia, along with Alexis Ohanian. Both of them have had a higher profile than me, and I admire what they've accomplished. I also think they've both been advocates for the technology industry, helping and advising others on how they can succeed in their own careers.
Jeff has been writing interesting posts about hardware and software for years, but the latest one struck me. It's called Learning on the Battlefield, and it has a lot of the same advice that I recently gave someone. In the post, Jeff notes that approaching your software career is like learning on the battlefield. It's like making weapons and coming up with tactics. You really need to test them on the battlefield.
Read the rest of The Battlefield of Your Career
I ran across an article, titled When to use CHAR, VARCHAR, or VARCHAR(MAX), which struck me as something I never do. I mean, I do use varchar (and nvarchar), but I can't remember the last time I actually created a char column. The article is worth a read, and it sets the stage for you to think about your database design process and the choices you make.
It's Friday, and if you take a few minutes and think about the last few times you've added columns to a table have you used CHAR as a data type? Or do you default to varchar of any size as a general rule?
Read the rest of Do You Still Use CHAR?
The cloud for computing is a fascinating structure. I know there are plenty of jokes about the cloud just being someone else's computer, and there are good reasons not to use the cloud. However, there are also lots of good reasons to use the cloud. Whether you choose to embrace it or avoid it, cloud computing is going to be a part of our careers for a long time. The use is growing, and more and more companies are shifting workloads to cloud services.
How that will evolve, especially for data-intensive systems, will be fascinating. One of the interesting changes that seems to be taking places is the growth of database and data stores that fulfill specialized roles for customers. Snowflake might be one of the most well-known examples, but there are plenty more. The number of offerings is growing, and perhaps this is another evolution of how the cloud will integrate into more businesses.
Read the rest of The Cloud as a Foundation
I received a report at SSC of a missing toolbar in the forums. I checked, and sure enough, the toolbar for formatting code and text in responses was missing. I submitted a high-priority ticket, mostly because I hate unformatted code. The site was still usable, but people would be posting stuff that just didn't look good. The developers got to work and had a fix, but it still wasn't right. The wrong toolbar was showing. This was a generic toolbar, not the one that we have had for years (this one). They produced a second fix that contained the right toolbar.
The developers are in and out of the SQL Server Central codebase, working for a few days on open issues, then busy with other work for a week or two. They work in a DevOps style environment, with changes run through a Pull request/peer-review process, then CI, then an automated deployment. They often can make a change and deploy it in hours, or even minutes.
Read the rest of Speedy Break Fix
In 2021, many companies had employees leave positions. The number of people resigning from their jobs was so high that the term "The Great Resignation" began appearing in many publications. A number of industries were affected, and there are no shortage of pundits and experts analyzing why. This happened primarily in the US, but the UK is concerned. This doesn't appear to be happening in Europe, but there are concerns in Asia and Australia as well.
The term feels misleading to me, as workers aren't forgoing work. Rather, they're changing jobs and looking for new positions, either at a better position in their field or leaving the field altogether. This is disruptive for firms, and costly, as finding new employees can be challenging and time-consuming. We also see a great loss to teams as the more talented employees find it easier to leave and replacing those people with less-skilled workers impacts their productivity.
Read the rest of The Great Developer Resignation
I saw an article about automating responses to security issues being a marathon not a sprint. The articles gives a few examples of different levels of automation response to situations, noting that each of this is a different level of maturity in the organization. At early stages, the response is mostly alerting a human to take action. Later, the automation will make some changes, but still defer to a human for more actions. The final example is automation handling most of the issue itself. Note, none of this means humans are unaware of what responses are being made.
The idea is that improving security with automation is something that takes place across time, as the organization matures and becomes more comfortable and trusting of automation. It's a marathon, where we push, but we know this will take some time to get to the end. It's not a sprint where we make a quick fix and get a result.
Read the rest of Think Marathon, not Sprint
I ran across an article on Zoom fatigue and new tech that was written in July 2020. That month seems like ages ago, and at the same time, not too long ago as the world seems to have still been in some sort of suspended animation for much of 2020 and 2021. However, one thing that has remained since that time is a regular stream of Zoom and Teams meetings that I attend.
I've been dealing with video meetings for over a decade. For most of the time I've worked for Redgate, I've had regular meetings every week with various people that are in Cambridge, UK. I'm in Denver, US, and while I enjoy going to England and visiting the office, that's a few times a year. Across the years, we've tried a number of platforms, and various different configurations, including a dedicated conference room with extra mics and sound-absorbing padding on the walls. None of this made for a great experience, though I have to say our adoption of Zoom across the last 2-3 years has been the best experience overall.
Read the rest of Making Video Chat Better
I would guess that many of us think we're good at our jobs, or at least that we do a good job. Whether you're a system administrator or software engineer, I'm sure you think you earn the compensation you get. I wonder if others in your company would see you this way? If they know what you do as a job, and understand the role, then you should hope they see you a good engineer/admin. Perhaps you should even ask a few people if they do.
There is a good thread about software engineers, talking about some of the things that make a good developer. It's from a longtime engineer, and the thread talks about the things that you might consider important in a good software developer. Actually, I think most of these items would apply to admins and even other business people.
Read the rest of Are You a Good Engineer?
DevOps has become somewhat mainstream in that many companies, perhaps most, are talking about it or trying to implement some of the principles behind this software development approach. However, even as I talk with people that practice DevOps, I find the idea of what DevOps actually includes can vary widely among developers, even inside the same company.
There are lots of definitions, and this one caught my eye recently. It says, among other things, that there should be "increased collaboration among not only developers and operations staff but users." I like that there is some emphasis on working together, which is often the hardest part of changing how we build software. I find that many customers struggle because they end up adopting the technology tools, but fail to change how they communicate with others. Culture change is hard, and DevOps requires a culture change.
Read the rest of Defining DevOps
I have seen a few people call for raising the RAM limit in the Standard Edition of SQL Server. In 2016, Aaron Bertrand voiced this, and for 2019, Glenn Berry asked that the limit be raised to 256GB. In the last newsletter of the year, Brent Ozar asked Santa for a 256GB limit.
I wonder how many of you would really take advantage of that. In the Azure SQL Database pricing table, to get beyond 128GB of RAM, you need to go to 32 cores. For Azure VMs, you need to purchase even more cores. AWS EC2 VMs require 32 cores to get to 256GB.
Read the rest of The Need for 256GB
I saw this study recently, where most people think they are better-than-average drivers in the US. I thought about it myself afterward, and I think I'm maybe average. I do pay attention most of the time, I try to be more careful when it's wet or snowy and give myself more time to slow down. I try to be aware of the other cars and obstacles on the road. I also usually have a drink (coffee, soda, water) with me, and I certainly mess with the music player. The Tesla reminds me constantly when I've strayed near a line, so a few months of driving seems to tell me I'm not great. Certainly not above average.
I would expect that most of us think we've done a good job with security for our systems. We don't expect to get hacked or deal with ransomware. I don't know how realistic that is, as a recent set of penetrations tests shows the vast majority of networks are vulnerable.
Read the rest of Most of Us Are Vulnerable
Every year I'm responsible for a portion of the Database Weekly newsletters. I used to do half, but right now Kathi, Grant, and I are on a rotation. Our boss puts out a schedule each year, which tells me the weeks for which I am responsible. Each January I then set up a recurring appointment for Mon/Wed/Fri on my weeks to remind me to curate articles and write an editorial.
Each year I also think about whether or not I can automate this. Last year I spent a few hours looking into the Exchange APIs for creating appointments, at the C# code, and PoSh (which is a nightmare). I spent a few hours, got frustrated, and gave up, just manually creating appointments.
Read the rest of It Isn't Worth Automating
At some point during my career sharing files and links in email became a security concern. There were all sorts of malware problems and issues that spread throughout organizations because someone unknowingly clicked on or forwarded a malicious item. At a couple of large organizations, we had to turn off email for days while we cleaned out systems. Fortunately, antivirus and other types of scanning software improved to prevent most of these problems.
It seems that not all developers learned the lessons of the past. Microsoft Teams has a few vulnerabilities inside it based on link sharing in the chats. One of the issues is that the preview features aren't well implemented, though Microsoft security doesn't think this is a problem as a user would notice the link would be incorrect. I think that's a huge leap and not a very safe one to make.
Read the rest of Teams Security Issues
In my life, I've mostly dealt with people that speak English. I tried Latin in high school and Japanese in university, but I wouldn't say I learned a lot. This past year I've been working on Japanese (konichi wa) and French (Bonjour) on a regular basis, but I'm still fairly uni-lingual. At least compared to many of the people I've met in my travels that speak multiple languages. I'm mostly just trying to read a bit, and that's quite hard.
In my career, I haven't met many technologists that only know one programming language. Even the SQL professionals know a little C# or a scripting language. Quite a few people know a bit about many languages, and some are very competent. I used to think it would be good to master one language before moving on, but I find that trying different languages can give you an appreciation for others, or even an appreciation for your primary one. Comparing and contrasting code might even help you learn faster.
Read the rest of A New Language
I posted a note on Twitter about some code. In this case, it was the start of a for loop, but with two things in it that I didn't like. The code is shown here:
for ($i = 2011; $i <= 2021; $i++) {
Both the starting point and the ending condition were problematic for me. This code is used in a scheduling application, to populate a dropdown that I used regularly. It's always worked, since about 2017 or 2018, and I had no reason to even think about the algorithm. I assumed the code was something like this:
Read the rest of Coding for the Future
LinkedIn sent me a prompt just before Christmas that asked me to write a post on my 2021 wins, using the #2021wins hashtag. I thought it was a neat idea, and I blogged on this topic before I left for my final vacation. I primarily had work items in there, though a couple of work-related things. To me, while 2021 was disappointing in some ways, it was still a great year for me in many ways, and I've tried to keep that in mind. Not focus on things that didn't go well but those that did.
It's the first week of the new year, and many of you are likely starting to look forward at work and in your career. You might have made resolutions, or maybe you are just re-focusing after the holidays. Either way, on this Friday I'd like you to think back to last year, focusing on how your career might have changed throughout the second year of the COVID pandemic.
Read the rest of What Were Your 2021 Wins
I was studying for the DP-900 exam recently and in one of the Microsoft resources, I ran across this quote: "Historical data is equally important, to give a business a more stabilized view of trends in performance." This was in context of looking at database performance, but it would apply to any part of your business.
The world has changed dramatically in the last two years. In early 2020, at this point, I was finalizing work and getting ready for a sabbatical from work. I managed to get the last sabbatical before the world shut down and most people didn't see the value of one during a pandemic. I returned from sabbatical, with a trip planned to the UK in early March that I canceled. Since that time, I can't imagine the quote above applies to the airlines' business. I've flown a bit across the last year, but rarely for work. I suspect most of their models and ideas about how to manage their business had to dramatically change.
Read the rest of Challenging Trends
The end of the year is when I do a little data cleanup. Not a lot, but some. Work slows down, with no major deadlines, so I'm able to spend a bit of time organizing myself. I usually go through my downloads and documents folders, deleting files I don't expect to use. I ensure that my jobs removing older log and temp files are running, especially on the laptops. A lot of disuse in 2021 had a few of them filling disks with more files than I had expected.
In a few positions I've had as a DBA, this was a time when we'd look to archive away some older databases, and even some data. It was rare, but always invigorating because we knew queries would run faster in systems when we could archive off data. I haven't ever started with clean databases in a new year, but in a couple of jobs we'd archive off all data that was more than one or two years old into another database. It was available if someone needed it, but the main OLTP databases would contain only one or two years' worth of data, helping queries perform quicker at the start of the year.
Read the rest of Data Cleanup
Today is my last day of work for 2021. I'm off to a family vacation for the rest of the week to enjoy time together and celebrate the New Year. I wanted to take a look back at the year before I leave and give you a few thoughts on various data-related topics.
The big news for those Microsoft data platform people is that SQL Server 2022 is coming. No idea when, and without public CTPs, I think it might be a 2022 H2 release, but Microsoft committed to the naming. There have been a number of talks, at Ignite, the PASS Data Community Summit, and the SQL Server and Azure SQL Conference. I had the chance to talk with Bob Ward and Pedro Lopes, and they are excited and proud of the upcoming release.
Read the rest of Looking Back at 2021
I work for a great company, one that I feel cares about me as more than a "resource", gives me opportunities (with responsibility and accountability), and has some nice benefits. I actually used one of those last week. I get a volunteer day every year, where I can spend time with a charity and get the day off. I used mine to volunteer for Habitat for Humanity in Denver.
During the last two years, Redgate has approached the pandemic and remote work with the idea that we need to be reasonable with each other. We need to understand that some employees might have more people in their homes during work hours and may need to provide care for others. They might have other challenges, or just be struggling to cope. We were asked to work with people, rather than against them, and try to support each other as best we could. I think it worked out well, and it showed us that company cares about us.
Read the rest of Wellness Week
We know that our organizations will adopt and use more devices over time. Given the growth of cheap computing, frameworks for managing devices, and the desire for more data, I expect some of those devices will collect data, or even contain databases. Azure SQL Edge use is growing, and we will see more devices that contain it (or another database platform), which means we have a larger attack surface area for that data.
There was a recent report on a vulnerability in edge devices used by AT&T that was detected as part of an attack. The attack used a known vulnerability based on default credentials. The vulnerability was fixed, but the patch required manual work. From various reports, it is unclear whether devices have been patched. It's also unclear if customer data was accessed. Here is one such report, but there are others, all with similar information.
Read the rest of The Challenge of Edge Security
I saw a note recently that Panasonic had a data breach. There was a time I had a number of Panasonic products, from music players to telephones. My wife had them as a client and got a few swag items in the early 2000s. However, I don't often buy electronic appliances, and I haven't seen the Panasonic brand name in some time.
As I approach the end of one year and the beginning of another, I find myself nostalgic for brands that I used to seek out. I read a biography of Akio Morita, founder of Sony a long time ago and I admired their products. At some point, I had a rack of Sony gear in high school, including the double cassette deck. I bought my first TV, a Sony, in college. I can't remember the last time I picked up or used something from them, having moved on to Apple for most of my music needs and Samsung for TVs.
Read the rest of The Tech of My Youth
Perhaps the best quote I've seen in a long time: "These kinds of attacks are common in smart contracts because many developers do not put in the legwork to define security properties for their code..." I'm sure that this would apply to many kinds of software, not just smart contracts.
This is from an article on a hacker that stole money by altering a smart contract. In this case, tokens used to replace parts of the contract overwrote other tokens, which allowed a smart hacker to change prices and make more money. Or steal it, with a contract change, I don't know that theft is actually the correct term.
Read the rest of No-so-smart Contracts
Today is the Redgate annual award event, and I'm honored to be hosting again. I've been asked a few times, and I used to look forward to the gathering in Cambridge followed by our annual Christmas party. Last year this was virtual, as it is this year, so I'm hosting again from my home office. Not what I had hoped would happen, but I'll have some fun. Last year I had a number of shirt changes between my parts, and I may do that again.
I know there are often annual sales awards in many companies, usually with some vacation or prize being given out, but not a lot outside of the sales department. In most of my jobs, we've usually had some sort of holiday party, but nothing else for most employees.
Read the rest of Acknowledging Success
There was a thread on Twitter on career longevity and fixing issues inside an organization. IT notes that often there is a problem and someone is assigned or hired to fix them. Often when someone starts to dig in, they can fix some things and reduce the pain. That's good, as things get better.
However, as you gain more knowledge of the issue, sometimes you realize there are more fundamental problems. If you try to fix them, you might make things worse. This could result in a repeat of the process of worsening and bettering of the problems over time. You might not make much progress if new people are constantly assigned when things get worse. If you stick with the person that is working on underlying problems, things get better over the long term. I don't know how often I find management willing to stick with the same person.
Read the rest of Worse Before Better
Most of us that work with SQL Server likely use either the Windows authentication or a user name and password when connecting to an instance in SSMS or ADS. It's how we've operated for years, and likely will for some time to come. If you connect to Azure cloud resources, perhaps you use some multi-factor authentication (MFA), but that's a minority of us.
If this article is a picture of the real world, far too few people are using authentication beyond passwords for many services. While plenty are using fingerprints, patterns, or face recognition on a mobile device, that's usually the extent to which they actually go beyond a password. I've actually started to see people using PINs on laptops instead of a password, which feels like a step backward.
Read the rest of Getting Beyond Passwords
One of the trends of the last ten years has been for many developers to try and avoid using a relational database where possible. Some look to NoSQL data stores, and others even consider flat file type stores of JSON or other formats that allow developers to work with speed and agility. Quite often it seems that applications grow to require some sort of relational store, often as an additional data store.
I ran across an article from a data science and analysis developer that is often performing work in R or Python on datasets. At some point, the post notes that when your dataset(s) become larger than memory, you might want to consider using a local database of some sort.
Read the rest of When to Use a Database
Yesterday was Thanksgiving in the US, a day where families typically celebrate together and many people give thanks for what is going well in their lives. It's also one of the days when more businesses and services are closed than any other day of the year.
This is the time of year when most businesses look forward and plan the future. It's also a time when lots of individuals may do the same thing, getting ready to make resolutions or attempt to change their lives in the new year.
Read the rest of Taking Stock of Your Career
We are seeing AI and ML becoming used in more and more industries, but one that seems to be a place where it is embraced with some success. From speech recognition and transcription to analyzing imaging, computers have helped medical professionals improve the care they give to patients.
Just as CAD has helped manufacturers, AI systems are being used in medical research, trying to model and screen medications to try and determine which ones are potentially useful in treating various diseases. We also have used them to better tailor treatments for certain diseases, like some cancers. It seems that applying computing against vast troves of data is proving itself beneficial.
Read the rest of A Strange AI Achievement
I am a big advocate of version control and testing for your software. The main reason is that modern systems are incredibly complex, and capturing what has been done (and what is happening), along with looking for potential issues, is important. It might be critical for ensuring you have success in building software.
Part of the reason is this post on software complexity. Now I don't think the complexity is killing software developers. I do think it adds to developers' stress, and in poorly managed environments, the complexity makes for a poor job experience for many. While I do think most people are smart, and they find ways to overcome this, they just don't enjoy the process.
Read the rest of The Complexity of Modern Systems
During the PASS Data Community Summit last week, Brent Ozar delivered the day 3 keynote, and it was great. I really enjoyed the format and topic, and I look forward to how someone else might be creative next year with the time. I'd certainly love to do the keynote, though not sure I'd be eligible. I'm hoping a few others get interested and compete for the slot in 2022.
One of the things Brent mentioned was about looking for skills to focus on, primarily for "builder," or those that develop new applications. He used a quote from Wayne Gretzky: "I skate to where the puck is going to be, not where it has been." The reference is to pick things that will be relevant in the future (where the puck will be), not where things are today.
Read the rest of Where is the Puck Going?
While talking with a client recently about their performance challenges, I was relieved to find that the database wasn't the problem. Instead, their API server was overloaded by the number of calls taking place in their application. While the database did provide the backing for the API calls, there was a fair amount of caching. However, as they'd moved to microservices, more and more of the interaction between modules was taking place as a network call to a single server, which became overloaded.Certainly, they could add more API servers, or break out different types of API servers, but each of these changes potentially has downsides. Data can become out of sync across two systems, which is why we often want a single database storing all data. Distributed systems are hard to architect and implement, and I suspect as more developers try to build them, we'll find more issues like this. Not that developers are doing anything wrong, but they often don't realize the challenges of building a distributed system when the workload scales.
Read the rest of The API Bottleneck
This is the second year that the annual PASS Data Community Summit has been virtual. Last year, as a pandemic gripped the world, I thought we might never see another one. I'm glad we have one this year, and I look forward to going back to a live, in-person Summit in 2022. That being said, this year's event has been better than I expected in many ways, mostly because of spatial.chat.
I can remember walking through the hallway of the Washington State Convention center and seeing Taiob Ali coming the other way. We stepped to the side of the hallway and had a chat among hundreds of people walking by. I thought about that in the Community Zone when he and I, with a couple others, were talking with each other in a group, away from others that were also in the room. We could see their avatars, but we were far enough away in the virtual space that our conversation didn't upset others.
Read the rest of Summit Reflections
When the COVID-19 Pandemic grew rapidly in early 2020, medical supplies were in short supply. Different areas and organizations struggled with different types of issues, and one of the higher profile issues was with ventilators. In March, the projections were dire and quite a few hospitals were worried about the supplies of these devices. This was especially disconcerting given how many people were being placed on them and how long it takes to produce them.
I met someone at that time that said getting a car company to produce these wasn't possible. The factories were specialized and converting machinery and people wasn't something that could be done in months. This was someone that claimed to have over 30 years of manufacturing experience.
Read the rest of Pulling Together
I've preached the value of Version Control Systems (VCS) for many years now. In various writings, in the talks I've given on development, CI, DevOps, and more, I always talk about the value of a VCS. One thing I often say is that if someone won't use version control, they aren't a professional. I don't say professional developer, professional DBA, or anything else. You're not a technology professional without knowing how to use a VCS in 2020. You certainly can't work for me if you won't use one.
I saw a nice, short post from Kevin Chant on the ways in which a SQL Server professional can benefit from a VCS. Many of us working with SQL Server aren't developers, we may not be some DevOps engineer. We might be someone that manages systems or handles backups. We might be someone that deals with complex HA, DR, or replication processes. We may be report writers or ETL developers.
Read the rest of Build the VCS Habit
I made a quick trip to Pensacola recently for SQL Saturday #884. I had a great time and consider myself honored to be picked to speak, as well as lucky that I can make the trip. I didn't have a smooth trip, as I wrote on my blog, but things worked out. These were really first world problems and minor hassles for me. When I wrote my blog, I was thinking about all the data I received, and didn't receive on the trip. Across the last few years, United (my primary airline) has enhanced their apps and services to share more data with travelers. I get text reminders two hours before flights, which is usually my drop dead time to leave the house for the airport. A good reminder for me. I get an announcement when boarding starts. I'm usually there, but if I'm late, or busy in an airport trying to find food, coffee, or a restroom, this is a great reminder. Read the rest of Data Challenges and Travel
I ran across a data breach at Dow Jones, the parent company of the Wall Street Journal newspaper. Apparently there was a database export that was stored on an Amazon S3 bucket. A security researcher discovered this and notified the company. Once the issue was corrected, the article linked above was published.
The details note that the security on the bucket was set to authenticated users, which is good. So, some security was set up. To me, if I were operating in an Azure or AWS environment, I would assume this meant authenticated users in my subscription/account/etc. In other words, my business partners and employees.
Read the rest of Who Are Your Authenticated Users?
Monitoring a database server is something that many of us know is important, but we often take the process for granted. Whether we've purchased a tool, like SQL Monitor, or we've built our own system, we often set up a watcher for our systems and rarely view the details unless something goes wrong. I'm not sure that's the wrong approach as part of the reason monitoring is set up is to allow data capture in the background and remove one more task from our daily workload.
Monitoring isn't necessarily simple, however, and while I still debate the best way to do this in many organizations, I realize that monitoring isn't necessarily something I want to build in house. There is enough work to just work with the data that other systems might output that I really want some other software in place that is built to perform monitoring for specific technologies. In reading about the complexity for the Stack Overflow monitoring systems, I realize that this can become very complex for a "set it up and let it run in the background" configuration.
Read the rest of The Devil is in the Monitoring Details
I haven't thought about this in some time, but Brent Ozar raises an interesting question: should index changes require change control? I've worked in an organization that didn't consider an index change to be an item that was presented as an official change. They were seen somewhat like adding new users or changing a firewall rule. These were ticket requests that could be triaged and made by an individual group without notifying the rest of IT.
I've also worked in organisations where an index change was viewed as a deployment and subject to change control procedures. Not that the network or security people cared about the details of the index, though the help desk might. These just wanted to be aware as any deployment change could affect their departments.
Read the rest of Are Indexes Actually Changes to the System?
A few weeks ago I gave a presentation at the 24 Hours of Pass Summit Preview. During my session, a demo broke and I had to ignore it. Later I found the issue and blogged about it. What was neat for me is that I didn't need to keep my code up or apps open, but I still found the issue quickly. I found it because I have a DevOps process that instruments and tracks everything.
This happened to me a few years ago. I was at a SQL Saturday and demoing one of the Redgate tools. There was an error and I couldn't move changes to a downstream database, which threw me off. That was the point of the talk. I decided to debug on stage (or behind the podium in this case), looking through the logs. Within a few minutes, I had found the issue and solved it. A real life DevOps story in action.
Read the rest of DevOps is Really Helpful
I think PowerShell is a great addition to the Microsoft stack. Given the previous versions of VBScript, Perl and Python ports, and more on Windows, PowerShell is a great improvement. Even for someone that spent a lot of time in the C Shell and Bourne Shell as a student and young professional, I think PowerShell is an improvement. There are certainly still things that make me crazy about PoSh, such as the -eq, but I'm getting more comfortable with the structures and flow.
In SQL Server, we had the old SQLPS module, which was, well, a start. Then we got the SqlServer module, which is better. However, the best thing I've seen for us data professionals is the dbatools project from Chrissy LaMaire (@cl) and company. To me, this is incredibly useful for anyone that wants to use PoSh with SQL Server, but it's really indespensible for SQL Server migrations from instance to instance. I'm not sure I'd even try anything else at this point.
Read the rest of A Great Case for Powershell
Every few years, or every year now, we see a new version of SQL Server proclaimed as the best ever. A few weeks ago, I saw a number of people leaving hte MVP Summit, rating it as the best ever summit. At SQL Bits or the PASS Summit every year, there's no shortage of people posting about that event being the best conference.
Is this a human view, that the latest one is often the best one ever if we enjoyed it? Perhaps we learn something and we're excited about the latest new SQL thing and so we're excited to proclaim it the best?
Read the rest of The Best Ever
I've been working through some of the GDPR legislation, trying to find ways to explain this more clearly to customers and ensure that our products make sense in light of this law taking effect. Redgate is focused in this area and not only do we need to ensure we are compliant, but we also want to ensure we are building tools that help ensure you are compliant.
In article 14, there's this text: "the controller shall provide the data subject with the following information... the existence of automated decision-making, including ... meaningful information about the logic involved." That sounds a little concerning for those of us that work with data. It's not always easy, but we can explain how a SUM or AVG function works, even with a complex OVER() clause and lots of joins and criteria.
Read the rest of An AI Loophole
My employer licenses software to users. Many of you might have SQL Compare, SQL Prompt, or one of our other handy products. If you do, you might have noticed that we have a Redgate login for you that shows your licenses and lets you activate/deactivate them. This was surprisingly a big project across the last year to streamline and smooth our licensing process.
Early on, I realized this was an issue in one company I where I worked. This was during the 1990s and I started working at a small company with a fancy imaging system. We had purchased software to receive all our faxes as images and file them in a digital system. I'm sure we had one of the smallest (and cheapest) installations of this software, which one of our executives had managed to negotiate. However, the exec had left the company and a few months into my tenure, I needed support.
Read the rest of "Who's Licensed?"
The cloud is a term that's full of hype. We hear from various media outlets all the time: the cloud is the answer, the cloud is cheaper, the cloud is the way of the future, the cloud handles your DR, the cloud managers availability, and more. Microsoft has been pushing the message of "cloud-first" (and mobile-first), which has many SQL Server professionals confused, concerned, or even angry. There are also plenty of professionals that dismiss the idea of cloud anything when it comes to data.
I've felt similar emotions, and certainly I have been skeptical of the cloud versions of databases. I remember the first cloud service, a key-value store, which seemed woefully inadequate for most purposes. Since they I've seen the Azure SQL Database grow, and many other products get released. Across that time, I've become more and more impressed with what Microsoft has done, and as Visual Studio Team Services has expended, I've come to really embrace and get excited by the cloud. It's still not something I'd always recommend, but I would always start there.
Read the rest of "The Cloud is Just a Tool"
A few years ago, I had a keyboard die. At the time, I needed something quickly and ended up with a Logitech wireless model that included a mouse. I'm not sure of which model, but I've ended up getting two or three more Logitech devices over the years. I think I go through a keyboard every 18-24 months, though the mice seem to last longer. For me, having devices available without cords is more important for the mouse than the keyboard, but since Logitech will bundle them, I get both at the same time. I'm currently with a K350 keyboard and an M510 mouse, both of which seem to hold up well.
However, I'm at home, usually working alone, so I haven't been concerned much about security. In corporate environments, I remember playing jokes by moving someone's mouse to my desk, and giving them a disconnected replacement, trying to mimic their movement. Or doing the same with a keyboard. That's a great typing challenge if you ever try it.
I thought about those times when I read this piece on the security of wireless keyboards, or maybe the lack of security. Apparently a relatively simple device can intercept and replace, or just record, keystrokes made on a variety of keyboards. These devices use their own dongles, not a Bluetooth connection, and security is non-existent. Perhaps I'll take one to the Redgate office this fall and see who's actually vulnerable.
Read the rest of "Keyboard Hardlines"
I saw a note from someone that had a database set up in a High Availability (HA) configuration for production. This person had received an 823 error on the primary server, but a failover hadn't occurred. This wasn't a critical error, but one that noted some anomolies in a few pages, which potentially could be fixed by the automatic page repair in SQL Server.
In this case, the individual would have liked to have had the system fail over, just in case there were a chance this would impact production. To make this happen, an alert on the error would be needed, which then forced a failover. This wasn't part of the native SQL Server configuration, and this individual was concerned. However, there are certainly cases where a failover might not be warranted when there is some sort of reaction such as Automatic Page Repair.
Read the rest of "Failover"
The march to the cloud is inevitable. No, I don't think all your services will be run from a cloud service, but I do think that some of them will be. Maybe 10% for your company, maybe 90%, but likely somewhere in between. In the efforts to reduce costs, and maybe more likely, the idea of shifting costs to expenses, I would bet that most organiztions will take serious looks at the cloud in the next decade for a significant portion of all new services. They might not move most applications, but they'll think about it.
I saw a note recently on Geekwire that Azure is estimated to exceed Amazon Web Services (AWS) by 2019. Most of us might not care about the race between these two tech giants to build large cloud infrastructures, but their capital expenditure is based on the idea that your organizations will want to use some of those services. They have good reasons to think this with surveys and conversations with many of their customers.
Read the rest of "The Relentless Cloud"
I was reading through a list of links for Database Weekly and ran across this script from Pinal Dave, looking for tables where the clustered index isn't the PK. It struck me that this is one of those facts I consider to be so simple, yet I constantly see people confusing. If you click the Primary Key icon in the SSMS/VS designers, or you specify a PK like this:
"The Clustered Index is not the Primary Key"
If you pay attention to security issues in software, you've probably heard about man-in-the-middle attacks. These can occur more frequently than you expect, though inside of a company, it's more unlikely that you'll experience one if you have some fairly basic security controls on your network.
However, if you invite someone inside of your network, such as a consultant of some sort, you should be more vigilent. There's a great post showing how someone can execute a MITM attack against SQL Server. It's in depth, showing exactly how packet captures lead to the ability to hijack a session and create a new login.
Read the rest of "Track Your Consultants"
Could a group of software developers make changes that fundamentally alter the way a software system should work without management being aware?
That's the question being asked of VW right now. Most people are skeptical, but I ran across a piece that wants to lend credence to the idea that a few software engineers acted with few people being aware. They did this, not because they wanted to defraud everyone, but they wanted a solve a problem that they couldn't do in other ways. They also didn't see the alteration of test results as much of an issue because they thought the tests were too stringent.
Read the rest of "Rogue Software Changes"
I grew up using a command line for all of my computing. I started with a Commodore VIC-20 as a youth, moving on to Apple IIs, TRS-80s, and then OpenVMS in university classes. It wasn't until I started doing graduate work that I started to enjoy graphics in the X Window System, though many of my windows just ran bash and korn shells on SunOS. Since that time, I've mostly made my living in Microsoft Windows and OSX, using graphical programs. I've still been quick to open a command shell at times, but not too often.
However there have been times when I've needed PuTTY to access remote machines, or I've had to struggle with some PowerShell remote commands. What would be nice is to have some of these tools on every machine. I love PowerShell available, but it's often a pain when I encounter a new machine.
"The Command Line is Back"
In July the T-SQL Tuesday theme was defaults. There were lots of posts about various items, but I found one by Kevin Kline interesting. Kevin noted that the settings for model are not necessarily what many of us would want in production.
Is that really the case? This week I wanted to take a poll and see if many of you actually do create defaults for your new databases by changing model that are different than what Microsoft has done.
Read the rest of "Adjusting Model"
I'm not surprised, and I had expected to read about something like this much sooner. Apparently someone at the ride sharing company, Uber, posted a security key online in a GitHub distribution. I assume this was some sort of code repository for Uber that may or may not have been supposed to be shared publicly, but having used GitHub, I could see someone making a mistake and accidentally putting private code in the public space.
Uber is worried as the key is a security authorization key used to access their databases. Someone apparently downloaded Uber database files and now Uber is attempting to track them down. The whole situation is a mess, but there are a number of problems here that we could learn from.
Read the rest of "Source Code Security" at SQLServerCentral.
I ran across a review of a developer's first year at a company. The company is endjin and the poster describes their first year as a "software engineer apprentice," which is an interesting way to look at one's early career. While it's not for everyone, I suspect that many developers in this business could actually use a bit of apprentice time to solidify their foundation of knowledge.
The post talks more about the way in which the work is done than the actual tools involved. A few are mentioned, but this is really a look back at the philosophical way in which the company solves problems. More importantly, it's the view of an apprentice learning a trade.
Read the rest of "Software Engineering in Practice" at SQLServerCentral.
It seems that software always contains bugs. No matter how much time and effort is spent building an application, there will be issues. Sometimes this is because of a lack of testing, and sometimes this is because of poor testing, but in any case, the expectation that we will test our code is becoming more prevalent as we depend more and more on computer software. Users expect our software to work.
It seems there is never enough time to properly test software after it is complete. Perhaps your deadlines are too tight, perhaps there aren't enough resources to devote to comprehensive testing processes, but it really doesn't matter. We will never have enough resources in our QA and testing teams to do as much testing as we would like. We need to expect this and find ways to raise the quality of our work, given those constraints.
Read the rest of "Testing" at SQLServerCentral.
It's the slow time of year. As I saw tweeted recently, "And now for the month when nothing gets done." I haven't usually felt this way at a few jobs as the budget and planning process has typically been a part of a busy month. As I've run SQLServerCentral, I've also been looking back and forward this month, trying to think about our industry and data in a long term way. I have found December to be frenetic as times as I try to get in some ski days while still getting work done.
However this year I've been in a situation that I think many people get into. I had a number of vacation days I needed to take, so I scheduled them a day here and there across the weeks. Between being off for Thanksgiving week, and missing almost the complete last two weeks of the year, I'm finding it a bit difficult to get into the swing of things as I have a few days back at work. It seems that work naturally slows down in teams as various people take holiday time off, but this is the first time that I've struggled to actually get my own work done as I'm in and out of the office on an irregular schedule.
Read the rest of "Holiday Struggles" at SQLServerCentral.
I ran into someone recently that told me they were scared at their job. This person had built a number of ETL jobs to move data around between systems. They were trained as a developer, but had a little experience, got sucked into working with SQL Server, and had made a career without really considering themselves a DBA. They seemed a bit bewildered that they hadn't messed up the data in any of their employer's systems.
I've met quite a few people like that, who seem amazed to be trusted to accomplish the work they do on a regular basis. It's good that many people can go through a career in SQL Server and successfully accomplish the things their employer needs done. However there's no shortage of "scary DBAs" out there that are in over their heads and do cause damage to data and systems.
Read the rest of "The Real Scary DBAs" at SQLServerCentral.
Most of us that are data professionals think the best way to make decisions is to use data to justify some course of action. We look for patterns in data, some guidance that the information we have will lead us to make the best choice for our organization. Google has talked about making data driven decisions as a part of their success and they think more organizations should do this. Any number of other companies also use data to power their BI systems and dashboards that help their employees make better choices.
That seems in contrast to this piece from the Harvard Business Review that says that great decisions don't start with data. It talks about using stories and emotions, with a few key facts sprinkled in, to help sell ideas and get decisions made. On one hand I agree that stories help to sell decisions, but I often have found that successful salespeople use this technique to deceive and convince by plucking emotional heartstrings, and using relatively little data.
Read the rest of "Data Decisions or Instinct?" at SQLServerCentral.
The DevOps movement is supposed to promote a closer collaboration between developers, testers, operational people, and really anyone else that must help software gets build and deployed. The developers need to coordinate their needs and requirements with operational staff. They should work early on with any staff performing testing to help them ensure that bugs are caught quickly and fewer slip through to production. At the same time, operational staff should be responsive and helpful in all the environments software moves through. Operational staff should help build and automate the configuration of development and testing environments, ensuring that systems are built in a repeatable fashion as well as the same way in each stage of the software development pipline.
This is my last day of work this quarter as I start my sabbatical at 5 o'clock today. I'll be gone for the next six weeks, away from SQLServerCentral, trying to improve and grow myself while staying as unwired as I can during that time. The rest of the staff here at SQLServerCentral will pick up the load, and I'm sure everything will be fine.
This is a bit scary for me. I've pretty much been employed full time since I was 17, only taking a few, short, involuntary stretches of unemployment in almost 30 years. Even during those times without a job I was focused on finding a job almost every day, so it felt as if I still had the job of finding a job during those times. I'm nervous about taking the time off, and certainly worried a bit about SQLServerCentral as I'm not sure I've been away from the site for more than a week since it was founded.
Read the rest of "Farewell" at SQLServerCentral.
I applied the Windows 8.1 update a few weeks ago and had some issues with my machine. Windows was fine, but I lost my SQL Server service. A few other users, including some of the SQLServerCentral community also had issues and sent me ideas, but their fixes didn't work for me. That was OK because the problems gave me a chance to use PoSh to solve a real problem. I'll be blogging about that in the next week.
However the 8.1 update has caused lots of issues, and Microsoft is acknowledging these problems. That's good, but the process gives me pause, and to a large extent, I think this makes more and more people suspect about all of Microsoft's patching processes. I bet there are companies that feel even more justified in waiting for SP1 for SQL Server 2014 before upgrading, even though there is a chance that the patch itself will cause problems.
Read the rest of "Patching Problems" at SQLServerCentral.
The Surface 3 was announced last week, and I was underwhelmed by the event, especially the presentation itself. The entire spectable didn't seem to compare to some other companies' announcements. I did, however, find myself thinking about replacing my laptop with a tablet that's lighter and allows me to forgo a keyboard. As much as I use a keyboard and depend on a responsive set of keys to get work done, there are definitely times when I'd like to just have a screen, and certainly a lighter form factor.
Today is Memorial Day in the US, and I'll be outside in a variety of situations today. Every year my son recognizes and remembers our soldiers at a Memorial Day ceremony with his fellow Boy Scouts. I like to capture some photos and video of the occasion, as well as track which Scouts are supposed to be there and have actually shown up. I can do it on a phone, but a laptop is too cumbersome. A tablet is actually the best option. It seems funny to use a larger device, but being able to capture and playback video on a tablet for others is much nicer than using a small phone.
Read the rest of "Laptop Dreams" at SQLServerCentral.
Have you ever had a development server crash? Have you lost work because of this? Had delays or had to recreate code? You shouldn't, or at least you shouldn't lose much work or time..
There was a time when I offered to manage backups on all development servers. This was in a large environment with hundreds of instances. I wasn't worried. I had scripts to do the work of setting up, running, and reporting on backups for instances. I knew how to deploy these scripts to hundreds of servers.
Read the rest of "The Development Backup" at SQLServerCentral.
Tim O'Reilly has written some great pieces over the years on many aspects of technology and change. I've followed him for years and I've enjoyed many of his thoughts and views on our industry. I think he's thoughtful and truly envisions the amazing things that technology can do to change the world, mostly for the better. I think it's worth randomly reading through (or listening to) some of his thoughts.
He recently wrote a long piece on the failures he's had in business with his company, O'Reilly Media. It's an interesting tale that looks at process, at mis-communications, and business. I think about my career, looking back on past decisions, looking at the way in which companies I've been associated with are run, and at the way that others around me have worked. I realize that there are many failures to go with the successes in my life. The piece made me take a good, hard look at my past, drawing parallels to the ways in which leadership at O'Reilly has worked as well as how it has come up short.
Read the rest of "Failure" at SQLServerCentral.
This essay isn't quite Safe For Work (SFW) and might offend a few people, so be warned, but I found it somewhat humorous. It's a look at all the insanity and problems with programming and software in the world. It certainly makes me glad that those of us working on software don't actually build tangible objects like bridges.
However the piece did make me think a bit. Why do we have so conflict and strange practices in many software systems? Why haven't we decided upon more good security practices that we widely disseminate? Why doesn't much of our code follow established patterns, like those that we might use to build a bridge?
Read the rest of "Why Don't We Have Better Practices?" at SQLServerCentral.
I was talking with some data professionals recently about training and the value of college, work, or some other method of entering the technology business. I have a few thoughts about the different ways of teaching people about this business, and I'm curious what you think.
What do you think of traditional computer science? Colleges have students study computer science theories, languages, and more. Build small software projects, write pieces of operating systems and more as part of their curriculum. Students also still have the requirements of other core classes like science, language arts, etc., but does this train them well? I almost think that students ought to be charged with working on real projects, perhaps open source projects, and tackling some small part of the system. Perhaps an older student needs to do project management work, or architectural specifications for less experienced students that are actually programming. Is that good preparation for the real world?
Read the rest of "Better Training" at SQLServerCentral.
There's likely a call for speakers open somewhere near you. When I was early in my career, I'd see these calls at large conferences like Comdex, TechEd, even the PASS Summit, and I'd think I'd never get the chance to present, with the best of the best speakers being accepted and no chance that I would be chosen. Things have changed in the world with the growth of SQL Saturday, there are a dozen open calls right now, all around the world, with many opportunities for new people to give a presentation.
When Andy Warren came up with the idea for SQL Saturday, the idea was that we would mostly find local speakers to help present at the event. We thought it might be expensive, and hard, to get the same speakers that present at larger conferences. This event would give many of the talented DBAs in local areas the chance to speak to larger audiences than they get at many user groups. SQL Saturday has been so successful, and the growth is amazing. We've started to see more and more well known speakers traveling substantial distances to attend events. That's great, but I do worry about the lack of local speakers at some events I've attended.
Read the rest of "Calling Local Speakers" at SQLServerCentral.
One of the most important things that you can do as a DBA, or really as any sort of system administrator, is back up your system.Ensuring that you have have backups, and of course, that you can restore them, is the number one priority for sysadmins. Everything else that you need to do is second to backups. After all, backups ensure you still have a system after a disaster. If you can't do that, then security, performance, features, none of that matters.
I have worked in large and small environments, and in all cases, I've assumed that as the DBA, I need to be checking that backups are occurring, and that I can restore them in case of any issues. Often this has meant I need to work more closely with others that have the actual responsibility for performing backups. This week, I'm wondering how many of you work in similar situations.
Read the rest of "Backup Responsibility" at SQLServerCentral.
There's a great quote I read, at the end of this article. It says: "...if you think that switching to NoSQL will just let you hand-wave away all of the challenges of running a database, you are terribly misguided." The context is that all too often people looking to move away from some of the hassles of working with RDBMS platforms, which includes working with the DBA, haven't completely thought through the issues.
I do think NoSQL has a place in the world. There are domains of problems that I'm sure Riak, MongoDB, and others, solve in a more efficient way than SQL Server, Oracle, MySQL, and other relational systems. I'm not sure what they are, and to some extent, I haven't seen good guidance on where particular platforms excel. Most of the articles and pieces on choosing NoSQL seem to be trying to sell me "why a particular platform can replace my other one", and telling me to add in things like transactions, but not explaining the drawbacks.
Read the rest of "No Handwaving Away the DBA" at SQLServerCentral.
Most of us have managers that are picked from the ranks of our technical staff. Often the most talented technical person, or most willing technical staff member, is promoted to manager, given a few hours of training on the various tools that he or she will use to deal with the administrative paperwork a manager fills out, and then cut loose to "manage" a group of people.
And they stink at it.
Most of them are bad. Not only are they bad managers, they are often bad leaders, and many find their technical skills suffer as they split their time with managerial tasks. That certainly puts them in a bad mood and creates worry about their future. It's not even just in the technology industry, as we find that many managers are bad, yet still languish on for years. This despite the fact that research shows they are costing their companies profit.
Career: an individual's journey through learning, work, and other aspects of life.
Vocation: an occupation to which a person is specially drawn or for which he or she is suited, trained, or qualified.
Which one describes your current employment situation? How do you feel about the technical work you do? A career is a good thing, and I hope that you all have long, successful careers, I hope you continue to learn and maintain a balance between work and your life outside of your job.
Read the rest of "Vocation or Career" at SQLServerCentral.
Many of the people that work in technology are interested in gadgets and hardware as much as software. Not all of us, but many of us have had side projects where we've melded hardware and software together. For those of us that are older, and started working with computers decades ago, that might have been the only way that we could get a computer system to work.
I'm glad that I still find people interested in hardware. My son recently built his own gaming computer from parts, and I'm hoping to distract him from some games and get him to help me build some robots with a Raspberry Pi computer and BrickPi add on board. However it's not just kids as I've seen a number of people at Red Gate working with hardware to build a variety of interesting projects.
Read the rest of "The Gadget Itch" at SQLServerCentral.
With SQL Server 2014 released, there's the temptation to upgrade for many DBAs. However the licensing costs and debatable improvements in the product will temper the DBA's enthusiasm with the reality of the ROI seen by management. While reading about the licensing changes, I also saw this note from Tom LaRock, where he wrote about the features most of us aren't using. It made me think about upgrades, and perhaps the strategy Microsoft is employing.
As Tom mentioned, the features not being used are Enterprise Edition features. This prevents many of us from upgrading to use them because Enterprise Edition is so expensive. Actually, even Standard Edition is expansive these days, given the per-core licensing, and I suspect lots of companies with SQL Server 2008, 2005, even 2000 are debating whether or not the upgrade is really worth the cost.
Read the rest of "The Subtle Push to the Cloud" at SQLServerCentral.
Would you post your password on a wall in your office? Of course not, because other employees, the cleaning crew, even guests walking around your office would be able to access your system with your account. When I read Brian Kelly's post on passwords in files, that's what I thought of. Sticking credentials in a file, where they're subject to any kind of search, is a bad idea.
However this happens all the time. Combine this with a few other "common practices" like using sa to connect to a database and building dynamic SQL, and you might as well just set blank passwords and invite someone to have fun with your database. It's sad that we continue to see these types of software development practices in 2014, and especially poor to see them from companies that sell software.
Read the rest of "Good Practices for Software Development" at SQLServerCentral.
It's over. There's no more MCM program or certification from Microsoft. The last MCM test was given last year and no upgrades are planned for SQL Server 2012 or SQL Server 2014. That's a little sad, though there are quite a few of our SQL Server professionals that can still proudly wear the MCM title for the rest of their careers.
The MCM tests were designed differently than all the other MCP type tests, requiring more thought and deduction, as well as practical skills. The lab in particular was daunting to many of the MCMs, most of whom would tell you about the difficulties in getting through the scenarios in the limited time alloted. All of the people I've talked to found the challenge refreshing and also informative, enabling them to learn a few things about their knowledge, even from the problems they didn't complete.
Read the rest of "Open Source the MCM" at SQLServerCentral.
I often talk with people about building their brands and finding a way to ensure they are a highly desirable employee. One of the ways that I think people can do this is with a technical blog about their career. Having a technical blog allows someone to show off their skills in a particular area. The blog doesn't have to be ground breaking work or extremely innovative solutions to complex problems. While employers need those people, they also need people that do solid work every day on regular problems.
An interview isn't a great way to find good employees. Many of us have had experience with either (or both) side of the interview table and realize that interviews aren't necessarily that helpful. If we bothered to track the impressions we make of candidates and compare that to the actual work they accomplish over the first year or two, I suspect we'd find that we have no evidence that we were making great decisions. The success of employees seems to be a bit hit and miss and not based on our choices in interviews.
Read the rest of "Due Diligence" at SQLServerCentral.
Throughout my career I've been looking to consolidate SQL Servers when I find them. The typical employer I've had usually grows their IT infrastructure over time and many projects look like this:
- Build or buy a software package
- Buy a new server
- Install SQL Server and one database for the application
Over time this means I find lots of individual servers running at much less than full capacity. That's something that DBAs like, because it means that we can handle the inevitable spikes in resource usage that our workloads will encounter. However that's not what the rest of the business, especially the financial management, wants. Underused resources mean money that isn't spent well.
Read the rest of "Consolidation Matters" at SQLServerCentral.
A friend recently was asked to give a presentation on their career to a group of 12 year olds. It was a challenge to engage the students, and my friend was surprised that very few of the kids were interested in technology. I was disappointed as well since I think this is a great career choice, and worth a little investigation, especially at that age.
It's hard to convince people to enter this business if they don't have any interest, but if they do, I'm wondering what you might tell them. Today I'd like you to think about your words of wisdom to someone interested in your job.
Read the rest of "Advice for Newcomers" at SQLServerCentral.
This is what you build to juggle 6,000 tweets a second. That's the headline that caught my eye and it's about the challenges of Twitter and the data that they handle. Twitter definitely has a tough problem, one that few of us have, but perhaps they can help us learn to better deal with our own data from their experiences on an edge case.
The story is journalistic, not so technical, but it is interesting. Twitter has struggled with a blend of data that is partially crucial and must be consistent now (usernames) and other data that can be a bit out of date (tweets). They also have lots of unstructured data (photo/video) that is combined with more traditional, structured data. They've used a few different database platforms to store this data and assemble it with their application. That's the same things that most of us also do when we deal with many different types of data.
Read the rest of "One Database To Rule Them All" at SQLServerCentral.
It's 2014. SQL Server 2000 is 14 years old, but there are still quite of you managing instances. SQL Server 2005 is 9 years old, and I'm sure more of you still deal with that version. I know because I work for a software vendor and I'm constantly asked if our software will run against those two versions of SQL Server.
For many of you, however, if you're managing a SQL Server 2000 instance, it might only be 9 or 10 years old. Your company might still have been installing SQL Server 2000 in the year 2005. The same is true for SQL Server 2005. I wouldn't be surprised to find companies still installing 2005 instances in 2008 or even 2009.
Read the rest of "How Long Before You Upgrade?" at SQLServerCentral.
I saw this post recently about security and preparing for a data breach. The title caught my eye because it implies that we're all doomed. Do the rest of you think that? Is it a question of when we'll have a security breach not if?
Given the headlines, the news we find out about companies not disclosing security issues, the back doors and poor code in much software, is it any wonder that people think it's a "when" and not an "if"? Given the lack of realization from many companies that suffer incidents that they were even attacked, perhaps that's an assumption worth making.
Read the rest of "If or When?" from SQLServerCentral.
Big Data is constantly in the news. We've been asked at SQLserverCentral to try and develop some articles, perhaps even a stairway to explain what Big Data is and how we might use it. I'm still trying to grasp the concepts myself, and unlike the amorphous cloud, I'm still looking for some good examples of what Big Data really is.
When I ran across this piece warning that Big Data isn't the final solution to all our questions in the world, I wasn't surprised. The piece notes that Google Flu hasn't been very accurate in its predictions of outbreaks. At first glance, this gives lots of credence to the idea that the good, solid data analysis and mining techniques we've used for years are just as good as any new Big Data fad.
Read the rest of "Problems with Big Data" at SQLServerCentral.
The Connect system from Microsoft doesn't work that well, but that doesn't mean we shouldn't continue to press Microsoft for change.
Read the rest of "Lobbying for Change" at SQLServerCentral.