Voice of the DBA: Recent Episodes
None
I’ll be at SQL Bits tomorrow, Saturday Jun 20, 2025 for the final day of the conference. I wasn’t selected to speak, but since I’m in Cambridge next week I came a couple of days early to stop by the event.
As this publishes, I’m probably just about to land at Heathrow and I am looking forward to a fairly quiet day in London.
Tomorrow should be fun. I have lightly looked at the schedule and I see a few AI sessions that might be fun to watch, but mostly I’m looking forward to catching up with friends.
If you see me, please don’t hesitate to say hi.
Are you looking forward to SQL Server 2025? Or perhaps you think this is just another release, or perhaps you are not looking for new features or capabilities in your environment. Maybe you don’t care about new things, but are looking for enhancements to features introduced in 2017/2019/2022. There is certainly no shortage of things that can be improved from previous versions (cough graph *cough).
I ran across an article on the five things that one person is looking forward to in SQL Server 2025. It’s a good list, and the things included make me consider an upgrade. Certainly, any improvements in the performance area, especially with all the investments made in Intelligent Query Processing over the last few versions, are worth evaluating. They might help your workload, or they might not, but if they do, then upgrade.
However, test, test, test. I can’t stress that enough. Test with your workload, not some random queries. Spend some time setting up WorkloadTools or find some other way to replay a set of queries from multiple clients to see if performance improves. It’s far too easy to look at a query in isolation and make a snap decision. With a load, sometimes performance looks different.
The HA improvements are also enticing, especially the idea of offloading backups more easily. Of course, this means you need to ensure you can and know how to, restore a complex set of backups in an emergency situation. Distributed systems are complex, and backups from multiple nodes (remember, you might get unexpected failovers) are a distributed system. Make sure you consolidate those, and plan for potential disruptions if your backup system/share/location is gone. Local backups are always nice, but Murphy’s law might cause you problems in multiple ways with multiple nodes and backups moving across them.
Again, test, test, test, and consider weird situations taking place. They will occur, and you should ensure your staff has a simple way to deal with them.
We’ve had a few SQL Server versions that leaped forward. SQL Server 2005 changed the paradigm, and I think SQL Server 2016 was another time of dramatic growth. Will SQL Server 2025 be one of those versions, or is it one that has a few incremental improvements?
Let me know your thoughts today.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I had someone ask me about using triggers to detect changes in their tables. This is the third post in the series. The first one
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
The SetupWe’re using the same table from the first post. This is the dbo.Customer table with a PK and 5 other fields. Here is the data in the table:
I had this trigger in the last post, and showed how it captured updates to the ContactEmail field.
CREATE OR ALTER TRIGGER Customer\_tru ON dbo.Customer FOR UPDATE
AS
BEGIN
IF UPDATE(CustomerName)
INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerName changed')
IF UPDATE(AddressKey)
INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.AddressKey changed')
IF UPDATE(CustomerStatus)
INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerStatus changed')
IF UPDATE(CustomerContact)
INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerContact changed')
IF UPDATE(ContactEmail)
BEGIN
INSERT dbo.logger (logdate, logmsg)
SELECT GETDATE(), 'ContactEmail updated from ' + d.ContactEmail + ' to ' + i.ContactEmail
FROM inserted i
INNER JOIN Deleted d ON i.CustomerID = d.CustomerID
END
END
This seemed to work, but did it really?
The ProblemLet’s illustrate the big problem with this change. I’ll run this code:
UPDATE dbo.Customer
SET ContactEmail = ‘andy@sqlservercentral.com’
WHERE CustomerID = 2;
If I do this, here are the results:
I get a NULL? Why, the original value is null and when I concatenate null with other values, I get NULL. Not ideal.
Let’s fix this problem. I’ll use a function to handle null values. Note, I need to do this for both the inserted and deleted tables. Here’s the new trigger.
```
CREATE OR ALTER TRIGGER Customer_tru ON dbo.Customer FOR UPDATE
AS
BEGIN
IF UPDATE(CustomerName)
INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerName changed')
IF UPDATE(AddressKey)
INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.AddressKey changed')
IF UPDATE(CustomerStatus)
INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerStatus changed')
IF UPDATE(CustomerContact)
INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerContact changed')
IF UPDATE(ContactEmail)
BEGIN
INSERT dbo.logger (logdate, logmsg)
SELECT GETDATE(), 'ContactEmail updated from ' + COALESCE(d.ContactEmail, 'null') + ' to ' + COALESCE(i.ContactEmail, 'null')
FROM inserted i
INNER JOIN Deleted d ON i.CustomerID = d.CustomerID
END
END
``` We can see this handles the null appropriately.
In this case I’ve chosen to replace a NULL value with the word ‘null’. This means something to me, but I could have just as well replaced this with “blank” or any other word. In many applications a developer might display a null value as a blank, so choose what works for you.
I’m also including an example to show this works for multiple rows. Here I’ll update multiple rows and we can see each is inserted into my log.
This is a good reason to audit certain activities, as people will sometimes make these mistakes and updates lots of data.
This trigger is slightly more useful, and handles the NULL cases, but it still isn’t perfect. Imagine I need to parse out changes, or generate the reverse transactions, or even search for certain changes. Stuffing a lot of data into a single field is overloading it, and making it less useful over time. What we’d really want to do is separate pertinent data into different fields. In a NoSQL world, we might do this by using a JSON schema to track the before and after.
We could do that here, just stuff JSON into the log message and read it back out and de-serialize it.
SQL New BloggerThis post modified our trigger to address a previous design problem: not handling nulls. We also showed how to test the trigger with multiple rows. This shows I’ve added knowledge to my skillset and can test what I’m trying to do.
Write your own blogs that might examine what you’ve done poorly in the past and how to fix the problems you’ve identified, even in simple code. Many of us do this a lot.
This was a 20-30 minute post for me. You could likely do it in a similar amount of time.
I had to make a few changes to a SQL Saturday event recently. The repo is public, and some of the organizers submit PRs for their changes, and others send me an email/message/text/etc. for a change. In this case, an organizer just asked for a couple of image updates to their site. I opened VS Code, created a branch, added a URL for the images, and submitted my own PR. After the build, I deployed it.
And it didn’t work.
I had a broken image. I checked the URL in code and realized I had “events” copied before the URL, which wasn’t valid. Ok, edit the URL to be correct and repeat: new PR, build, merge, deploy.
And it didn’t work.
I was looking at the code live on the site, the code in the repo, and I was trying to reconcile paths and file names and keys and values and a few other things.
I realized the world for a developer hadn’t changed a lot, and in fact, I was in the age-old loop: deploy, patch, patch the patch, fix the patch for the patch, and so on. I don’t even know that I could have gotten better here with testing, as these were one-off data changes that affected the site for users. If I enter the wrong data, it’s wrong. I can’t easily test for this.
I have written code that was wrong, and a few simple tests would have caught my issues. I’ve also written code that isn’t easy to test. If I am adding or changing data, it’s hard to test that. Often, I might do some copy/pasting between the code and the test to generate the test. If I’ve typoed something, the typo continues through the test (in some cases). Even using a code generator or an AI to produce the INSERT or UPDATE code might not solve the problem. They might read my typos in a prompt.
One of the best things to help code quality in the last few decades is continuous integration (CI), where we have automated systems that compile code, test it, and run it. It’s not perfect, but it does help reduce the silly mistakes many of us likely make every day when writing code. These can’t prevent typos and issues, but if we are testing intermediate systems, hopefully somewhere along the way, a human or AI agent tries to verify that the things we were typing exist and can catch a typo.
In this case, I had to find where I’d mistyped the line and realized that I had the path wrong. The image was in a subfolder and I needed to add that to the img url.
Working with data is hard, and it’s a constant source of simple mistakes. I don’t know we’ll ever get away from patching the patch when data manipulation is involved.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I’ve been very happy with Docker Desktop for years, running it on both laptop and desktop. However, a corporate decision was made to move to Rancher Desktop, so I now have an unexpected “opportunity” to learn something new.
Here’s a short post on how things went on the desktop and laptop.
Getting Rancher DesktopI had never heard of Rancher. I’ve met a number of Linux/Oracle people using Podman, but not Rancher. You can download Rancher from rancherdesktop.io. This is a project from SUSE, of the Linux distribution fame, and one of their many projects.
The install is a next/next/next standard Windows MSI, though once installed, I found I needed WSL2 on my desktop. On my laptop, I ran wsl –v and saw this.
On my desktop, I installed this first, and tried to get things going. It installed, downloaded some Kuberbetes things, asked me to use containerd or dockerd (I chose the latter), and then was running. Once the engine was up, my docker commands worked and I could start containers.
However, there was a problem in that I had a volume (unnamed) that a few containers were using. However, after uninstalling Docker Desktop, I couldn’t find the volumes. I’m lightly concerned I’ve lost some data, which isn’t the end of the world, but it annoying.
Learning From My MistakesOn my laptop, I decided to try and make this smoother. First, I uninstalled Docker.
Next I ran the Rancher install.
Once this installed, I started the desktop program. It again downloaded some Kubernetes components and then seemed to be up and running.
I tried Docker components after starting the Rancher Desktop (RD), but before I realized it was downloading stuff. That was the docker error shown first below. The second one was once the online status was shown in RD.
First TestsI have a few containers set up to run SQL Server with docker compose files. I have a batch file I double click for “docker compose up” (and another for down). I clicked one and saw this: images downloading.
I assumed an image store would be an image store, but apparently not. Rancher must use a different place. That’s an interesting thing I need to check. Do I have extra images laying around. I think my Docker Desktop was using containerd, so maybe that’s part of this.
I could connect from SSMS fine and I could see my container running in the desktop (I had to switch away from the Containers item and back).
I also checked some docker commands and they seemed to work. I could get a list of containers, and apparently Rancher runs a bunch itself.
I could also see logs from my container, which are handy at times when I need to try and debug an issue.
SummaryI have to admit that after a week, I’m still nervous. Losing a volume, whether it’s really gone or I just can’t find it, is disturbing. I hate losing data.
Rancher seems to work fine for the basic things I do with containers, though the interface feels incomplete and simple. I can’t set hardware limits, it isn’t an active interface, and I feel like I’ve lost a lot of options. I didn’t really use more, but I still feel some loss.
I haven’t heard internal complaints from anyone, so I’m assuming that most container based things still work. We’ll see how I like this across the next month.
I can’t believe I’ve been at Redgate long enough to get a third sabbatical. I’ve very lucky to have this job, still enjoy it, and get the benefit. I’ve scheduled it from Jun 30 – Aug 8, and I’ll be gone from work during that time.
The idea of the sabbatical is to get away from work and recharge. It’s an extended break, and while some people use this to further their career in some way, others just try to get away from their daily life.
I wrote about a bunch of projects I had during my first one. The second one was less planned, but I also had a quick review of the time away.
For this one, I have no great plans right now. In fact, life has been so busy this year, I haven’t even had time to think about what to do, but I didn’t want to delay things, so this was the best time to take it.
I started this post to get me to at least lightly focus on the time and what the possibilities could be. For now, I have a few things to do:
- Work with a contractor to replace the covering on our riding arena
- Rebuild a better generator house and re-wire a circuit
- Replace some damaged fencing
- Rebuild 2 horse feeders
- Organize the garage a bit more
Don’t let someone else’s urgency becomes your emergency. In fact, don’t be governed by the urgent of any sort. Focus on the important. The urgent is a tyrant. – from Excellent Advice for Living**
I try to set my life up to be fairly relaxed. A little chaotically busy, but relaxed. I try to stay ahead of work, plan things, get them prepped, and beat my milestones, at work or at home.
However.
As some people might say, life happens. Others might use a different 4 letter word, but I’ve liked life. As John Lennon says: Life is what happens to you while you’re busy making other plans.
I do try to help others and accommodate them, but another’s emergency isn’t mine. I am here for support, and to listen, but encroaching on my wallet or my time is something I have to choose to give you. Sometimes I might, but sometimes I might not. Ultimately. I don’t want to get jerked around by others, at work or in life.
I try to remember what’s important, which might not be someone else’s thing. This might mean previous plans take precedence. Or it might mean that I decide to help with your emergency. The important thing is the important thing.
I’ve been posting New Words on Fridays from a book I was reading, however, a friend thought they were a little depressing. They should be as they are obscure sorrows. I like them because they make me think.
To counter-balance those, I’m adding in thoughts on advice, mostly from Kevin Kelley’s book. You can read all these posts under the advice tag.
When talking about DevOps, the goal is to produce better software over time. Both better quality as well as a smoother process of getting bits to your clients. There are a number of metrics typically used to measure how well a software team is performing, and one of the things is Change fail percentage. This is the percentage of deployments that causes a failure in production, which means a hotfix or rollback is needed. Essentially we need to fail forward or roll back to get things working.
For most people, a failed deployment means downtime. I’ve caused a service to be down (or a page or an app) because of a code change I made. This includes the database, as a schema change could cause the application to fail. Maybe we’ve renamed something (always a bad idea) and the app hasn’t updated. Maybe we added a new column to a table and some other code has an insert statement without a column list that won’t run. There are any number of database changes that might require a hotfix or rollback and could be considered a failure.
However, some people see an expanded definition. If a service is degraded (slower), is that a failure? Some people think so. If we change code in a database (or indexes) and see performance slow down. In that case, is this a failed deployment? Customers would think so. Developers might not like this idea, at least not without some sort of SLA that might allow for some things to be a little slower. After all, slow is still working, right?
What if I don’t notice a problem? Imagine I add a new table/column, and the app starts accepting data and storing it. What if we are supposed to use this data downstream, and we don’t notice it is being aggregated incorrectly by a process until many days later. Perhaps we’ve performed some manipulation or calculation on our data and the result isn’t what we wanted. It might not be incorrect, but maybe it’s ignoring NULLs when we want NULLs treated as 0s.
Is that a failure? If I deploy today and Bob or Sue notices next week that the data isn’t correct, that’s a failure. I don’t know I’d count downtime from today until next week, but from when Bob/Sue files a ticket, the clock starts on calculating the MTTR (mean time to recovery).
I don’t often see database deployments failing from the “will it compile on the production server” standpoint. Most code gets tested on at least one other system, and with any sort of process, we catch those simple errors. More often than not, we find performance slowdowns or misunderstood requirements/specifications. In those cases, some of you might consider this a failure and some may not. I suppose it depends on whether these issues get triaged as important enough to fix.
While I might have a wide definition of deployment failures for most coding problems, I don’t for a performance slowdown. Far too few people really pay attention to code performance and are happy to let bad code live in their production systems for years.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I’m traveling this week to various customer sites. This is a hectic week, with trips to Norfolk (VA), Nashville, and St Louis. Three customers in three days. This is a rough week, with meetings during the day, flying in the late afternoon/evening to a new city, and a repeat again.
As a result, you get What Keeps You Employed? to reread.
I had someone ask me about using triggers to detect changes in their tables. This is a second post looking at triggers, in this case, modifying my trigger to detect more changes and using that information.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
The SetupWe’re using the same table from the last post. This is the dbo.Customer table with a PK and 5 other fields.
In this case we want to track the changes to an email and capture what those changes are. In other words, if I update the email from ‘sjones@sqlservercentral.com’ to ‘steve.jones@red-gate.com’, I want to capture
- That the Contactemail was updated
- The old value was sjones@sqlservercentral.com
- The new value is steve.jones@red-gate.com.
To do this, let’s modify our trigger. We can still test if the field is updated with UPDATE(). We did this in the last post.
This will let us know that the column changes by returning a boolean. If this is true, we want to insert the new values into our logger. We also want to capture the old value. These values are stored in the inserted and deleted tables, which are available in a trigger. I’ll use a join between these on the PK to get the same data from both.
I’m also using a query with these tables because more than one row can be updated and we want to capture all the changes.
Here is my new trigger, with the OR ALTER added to the code.
CREATE OR ALTER TRIGGER Customer\_tru ON dbo.Customer FOR UPDATEASBEGIN IF UPDATE(CustomerName) INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerName changed') IF UPDATE(AddressKey) INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.AddressKey changed') IF UPDATE(CustomerStatus) INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerStatus changed') IF UPDATE(CustomerContact) INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerContact changed') IF UPDATE(ContactEmail) BEGIN INSERT dbo.logger (logdate, logmsg) SELECT GETDATE(), 'ContactEmail updated from ' + d.ContactEmail + ' to ' + i.ContactEmail FROM inserted i INNER JOIN Deleted d ON i.CustomerID = d.CustomerID ENDEND
This is mostly the same code, but now I’ve changed the last conditional test. If the email is updated, I want to query the inserted and deleted tables and create a log message. This is the type of thing I’ve seen often in systems, and while it’s not a great pattern, it does let me capture some information. There are some problems with this, I’ll discuss in the next post.
We can see below that when I run this code, I get the updated captured and logged.
This post has introduced a few new things, the inserted and deleted tables, which I didn’t discuss in the last post. However, they are useful when you want to capture information affected in triggers, which can be more than one row. Using these tables helps you set up triggers that handle multiple changes.
There are problems with this trigger, mainly with NULLs, potential performance, and architecture in what is captured, but we’ll address those in the future.
SQL New BloggerThis post looks at enhancing a previous post and providing more information. You (hopefully) learn from your work, from both feedback and experiments, and you should modify your thinking and work. This shows how I’ve adapted something I did previously, which is a skill we all need.
This was a 20-30 minute post for me. You could likely do it in a similar amount of time.
I started a short thread on Twitter/X and Bluesky recently after leaving the Tesla at home recently. A few people asked me about it, so I decided to do a short update, as this is my computer on wheels.
This is part of a series that covers my experience with a Tesla Model Y.
A Busy WeekendA few weeks ago, we traveled to Greeley, CO to coach in a tournament. My wife and I coach two teams this year with our daughter and it’s very busy. This particular weekend, we were coaching the afternoons on Saturday and Sunday and Monday morning in Greeley. The catch was our second team had a Sunday morning tournament in Golden.
We had booked a hotel in Greeley and decided to take two cars as we knew the Sunday morning event might run long and we needed to be back in Greeley. Our schedule of places, times, and drives was:
- Saturday 11am – practice (17 miles from home)
- Saturday 2pm – Greeley (72 miles from the gym)
- Sunday 7am – Golden (64 mi drive from hotel)
- Sunday 2pm – Greeley (64 mi from Golden)
- Monday 3pm – Leave Greeley for home (83 miles)
Around this, there likely are a few short trips of a few miles. Dinner, etc., so these are estimates.
The other caveat on this weekend was the weather was predicted to be in single digits (in the F scale) overnight. Either positive or negative, but cold. The highs were 11F Sat, 15F Sun, and 8F Mon.
Planning the TripWe already knew we needed two cars, so the question was: is the Tesla one of the two cars?
Some snow was expected, and I didn’t worry, but my wife had a bad experience heading to the mountains one weekend in heavy, wet snow and the Tesla defroster didn’t work well. Neither did the wipers.
That was a minor concern for me, especially as I learned how to get hot air rather than the default cool air for the front defroster.
The bigger issue was time. In trying to get up to Greeley, we would be tight on time. I’d be asking to go over 200 miles without a charge in very cold weather, across 2 nights.
Or we’d be looking to stay up late one night to charge at the Loveland supercharger, which is a 20 minute (each way) drive from Greeley. There are some chargers in Greeley on a map, but when we tried them last year we found:
- 1 didn’t work
- 1 was very slow (level 1 slow)
- 2 were busy
These were separate locations in the town, where businesses had set up fast charging. This isn’t a knock on Greeley, it’s just the reality of EV charging in some places. This is an unfamiliar place and I don’t trust most charge stations owned by various companies.
At least not when time is a factor.
The Final DecisionI didn’t want to be stressed about charging, so we decided to leave the Tesla home. It wasn’t that important, certainly not to worry about charging time or having enough charge in a very cold environment. I took our old Suburban and my wife took her diesel truck.
There ended up not being much snow, just a few flurries, but it was very cold, and we were racing from place to place. In coming back Sunday to Greeley, we had a 10 minute cushion according to Google Maps and lots all that trying to get a cup of coffee and some lunch from a stop. Even if we’d have aimed for the Supercharger for 10 minutes, they don’t have food service nearby. There is a hotel at the supercharger, but they’re a) slow and b) expensive.
I’m happy with the decision. I missed driving the Tesla, but it made sense to skip it for this trip.
SQL Server on Linux was released with version 2017. Since then, I’ve seen some deployments of SQL Server on Linux, but many of the customers I work with still deploy SQL Server on Windows. While there are limitations and unsupported features, most of what we need is available in SQL Server on Linux.
I assume most of you out there work on Windows machines against Windows servers. Maybe some of you run containers, but that’s likely a minority. Windows seems to have won the desktop and for most of us running SQL Server, the server room as well.
However, if you use containers, you likely use Linux ones since SQL Server isn’t supported on Windows containers. I know I do, and I like them, but overall, I find I need to know very little Linux to do my job, or even work with the containers.
I like Linux. As someone who learned Unix early on and installed Linux 0.8, I thought at one point I’d spend most of my career in that world. Especially as I worked with DOS and Windows 3.1 in corporate work and found them much less capable. I still remember writing grep.bat and awk.bat files to duplicate some of the things I did in Unix on DOS machines.
For doing database work, most platforms are ported to Windows, but even if you connect to an Oracle/PostgreSQL/MySQL/MongoDB/etc. system running on Linux, do you need much linux? I find that ls, pwd, and cat get me through most of the things I need to do. When there’s something more complex, like sudo systemctl restart mssql-server, there are plenty of code snippets in the docs or some website. These days, you could even ask an AI how to do many simple tasks.
If you don’t use Linux, then you don’t need any, but if you deal with any sort of system running on Linux, how much is important to know? What’s your top ten list of things a newbie should learn? Let us know today.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Life gets better as you replace transactions with relationships. – from Excellent Advice for Living
This is incredible advice. I think that much of the complaints about the US from the rest of the world is how transactional we are. Whether this is with stuff we buy or neighbors or how we eat dinner or how we treat sports, we’ve often become transactional. What’s the best, fastest, easiest, most convenient, etc. We often want a quid pro quo for things we do.
When you spend more time with others, when you value the experience and what you get from it, life is better.
It can be harder, slower, etc., but I think it’s better.
I’ve been posting New Words on Fridays from a book I was reading, however, a friend thought they were a little depressing. They should be as they are obscure sorrows. I like them because they make me think.
To counter-balance those, I’m adding in thoughts on advice, mostly from Kevin Kelley’s book. You can read all these posts under the advice tag.
I don’t know how many of you will be disappointed or impacted by this, but Azure Data Studio (ADS) is being retired, as of 6 Feb, 2025. It will be supported for a little over a year, until 28 Feb, 2026. On one hand I’m not surprised, and on the other, I’m a little shocked by this.
I have written a number of articles on ADS, and shown how things work, as well as pointed out a number of things that don’t work well in the product or its extensions. These pieces have gotten a number of reads, and people have commented on them, so I wonder if there are a lot of you that are upset by this. Is this going to change the way you work? I will say that it will lightly change my work, as I do use ADS to connect to PostgreSQL, but not so much for SQL Server.
I have tried to use ADS, but I just don’t like it. I don’t have a good reason, as it does a lot of what I need from a query tool. I think the port of the query and result experience from a real app like SSMS or Enterprise Manager or even isql/w is just a worse experience. I don’t like the ADS interface and it’s annoying to me.
I suspect that many others feel the same way (other views from Deb and Kevin). They don’t like the ADS experience and prefer SSMS or some other tool. I know there’s been no shortage of complaints over the years about, and finally MS has listened. From first trying to get everyone to leave SSMS to forcing people to install ADS alongside SSMS and now to finally retiring the tool. I think it’s a good decision as people don’t want to lose SSMS and it’s hard to maintain two tools.
We will still have VS Code, which I use often for other purposes. I haven’t spent much time with the mssql extension, but I need to as it’s been updated as of a few months ago and supposedly works better now. We’ll see.
In the meantime, I won’t mourn ADS. It was a tool that had potential. I liked the idea of notebooks, I liked the fast startup. I just wish it were better implemented as a run-a-query-and-get-results application. I wish we had a cross platform editor that was simple and fast, but not one based on VSCode. One that’s written to just manage queries. Maybe they’ll rewrite isql/w in a modern way and port it to Linux.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I wrote about getting the Redgate Test Data Manager set up in 10 minutes before, and a follow up post on using your own backup. One of the things I didn’t show from my own database was that it had no FKs, so the subsetting didn’t quite work as I wanted.
A previous post showed how add starting tables for the subsetter to look at, however that didn’t get me a good data set for testing. This post continues looking at the subsetter by adding manual relationships to our configuration.
This is part of a series of posts on TDM. Check out the tag for other posts.
Declaring a Relationship in the Options FileIn my previous post, I’d picked a starting table and had reduced the dbo.players table from 16564 to 1800. However, I only had player information. If I query my subset database, I see there is a player, but I have no batting statistics for this player.
This is because my table has no declared FKs in it. If I check the dbo.batting table, I can see only a PK.
Let’s fix this.
Declaring Manual FK RelationshipsIn the options file documentation, there is a section that notes manual relationships can be declared with a key called “manualRelationships”. If I copy/paste the example section into my options file, I’ll see this:
I don’t have a SourceTest table, so let me edit things. I’ll set a relationship between dbo.players.playerID and dbo.batting.playerID. This gives me the following in my options file.
Before I run my subsetter, here are the row counts by table.
I’ll re-run this subset command, which includes my option file at the end.
rgsubset run --database-engine=sqlserver --source-connection-string="server=localhost;database=BB\_FullRestore;Trusted\_Connection=yes;TrustServerCertificate=yes" --target-connection-string="server=localhost;database=BB\_Subset;Trusted\_Connection=yes;TrustServerCertificate=yes" --target-database-write-mode Overwrite --options-file E:\Documents\git\TDM-Demos\rgsubset-options-bb.json
When I do that, I know see these row counts. Note I now have batting rows.
My player query won’t work, so I still need to declare another relationship with the dbo.teams table. That is shown below:
I can re-run the same command above, and then I see this set of rowcounts (original on left, subset on right).
There is teams data, and if I re-run my queries from the top, I can see stats now.
Now I have a dataset that I can perform development work with in terms of players, teams, and batting.
I can also add more relationships as needed, for example, I’ll add this section to include pitching, batting post, and fielding. Here’s my complete options file:
{ "jsonSchemaVersion": 1, "startingTables": [ { "table": { "schema": "dbo", "name": "players" }, "filterClause": "birthState = 'CA'" } ], "manualRelationships": [ { "sourceTable": { "schema": "dbo", "name": "players" }, "sourceColumns": [ "playerID" ], "targetTable": { "schema": "dbo", "name": "batting" }, "targetColumns": [ "playerID" ] }, { "sourceTable": { "schema": "dbo", "name": "batting" }, "sourceColumns": [ "teamID", "yearID", "lgID" ], "targetTable": { "schema": "dbo", "name": "teams" }, "targetColumns": [ "teamID", "yearID", "lgID" ] }, { "sourceTable": { "schema": "dbo", "name": "players" }, "sourceColumns": [ "playerID" ], "targetTable": { "schema": "dbo", "name": "battingpost" }, "targetColumns": [ "playerID"] }, { "sourceTable": { "schema": "dbo", "name": "players" }, "sourceColumns": [ "playerID" ], "targetTable": { "schema": "dbo", "name": "pitching" }, "targetColumns": [ "playerID"] }, { "sourceTable": { "schema": "dbo", "name": "players" }, "sourceColumns": [ "playerID" ], "targetTable": { "schema": "dbo", "name": "fielding" }, "targetColumns": [ "playerID"] } ]}
After re-running the subsetter, I have these row counts. Note there are rows in all the tables defined in the options file.
I can keep adding in more tables as needed to ensure the subsetter can walk down the data relationships I need in my database to produce a useable dev/test dataset that’s smaller than production.
TDM can help your devs build better software and with the subsetter, this can create lots of agility to ensure the data you need to accurately build this software is available.
Give TDM a try today from the repo and a trial, or contact one of our reps and get moving with help from our sales engineers.
Video WalkthroughCheck out a video of my demoing this below:
I had to demo the Flyway Autopilot system recently and created a GitHub Actions runner as a part of that. This post documents how this went.
First, if you go to the settings in a repo and click the Actions area, you see a Runners item. Click that. Notice I have no runners.
In the upper right corner, I can click a button to create one.
This gives me the instructions to get a new one. Note, these are PowerShell commands, and the first command doesn’t quite work right. Still, this is what I need.
I opened a CMD window and stared running these. Note, I need to repeat the change directory.
Now start PowerShell as the next commands are PoSh ones. When I copy the next command, it starts downloading a zip file. As of this writing, this is a 600-ish MB file.
Once this is done, you can run the next commands, which unzip and configure this. For the config, I just hit enter as I don’t have multiple groups or tags, and I leave it named as my machine. I also don’t bother to run this as a service.
The last command runs the runner agent.
If I go back to the Runner screen in Settings, I see I have an idle agent set up.
And that’s it. If I pick one of my workflows in the Actions tab, I can run it and I’ll see the job started in my runner folder. Here are my actions with the Run workflow button on the right.
If I click this, I see the job start in the CLI.
If I get back to the Actions, I’ll see things in progress. As you can see, I was slow here.
That’s about it. Now I can run local automations in my repo that connect to things like local databases, which can be handy.
Video WalkthroughI’ve got a video of this process if you want to watch it.
I saw an interesting thread recently in the SQL Server Community Slack where someone posted about extended events (XE). They were asking about whether XE would have a problem with a situation. The problem wasn’t so interesting, but a quote from one of the responders was. The quote was:
The best time to have learned Extended Events was ten years ago. The second best time is today.
I love that, and I tend to agree. If you need to trace what is happening inside your SQL Server, you need to learn how to capture information with Extended Events. That’s the best way to dig into the details of how queries affect your system.
It’s also hard. I know that whenever I need to use it, which is rare, I have to dig through some articles and docs to understand what thing I need to do. Even having some scripts hasn’t helped because it’s a sufficiently complex system that unless I use it regularly, I forget how all the filters, targets, events, etc. work.
On one hand, I think it’s amazing, and on the other, it’s too hard to use. Even when I try the Extended Events profiler, it’s so different from Profiler that I find myself getting frustrated at times trying to dig through the information.
I am curious how many of you think XE is easy to configure and if you use it often. What are the places it works well? For those of you that don’t use XE or haven’t learned, why not? Do you not have to trace what’s happening with queries in some detail? Or do you have another way that you dive deep into your system? Or do you not have the need?
If you do want to learn more, we have a short Stairway Series on Extended Events to help you get started, as well as a few other articles. If you’re an expert, we’d love a few more on using XE in specific situations.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
There’s an interesting piece at O’Reilly this week titled The End of Programming as We Know It. That’s actually a good title, but it doesn’t mean the end to programmers, developers, software engineers, of whatever people call themselves.
The piece looks back at history, starting with physically connecting circuits to program and moving to switch flippers, cards, compiled languages, and on to the web and mobile systems. Each of this brought more and more programmers into the industry because there’s no end to the software needs of the world and it’s hard to write good code.
I like that the web was seen as the end of programming as anyone could easily build a minimal application to share things. Even frameworks like WordPress let novices create applications that do all sorts of useful things. However, we still need programmers. Look around at how many WordPress consultants are willing to help you.
That’s because there’s a difference between configuring something off a shelf (or from a digital store) and actually having a working application for your situation. There’s no end to what people want to build, and really, they need someone to build it for them. Or at least modify, improve, or re-implement the proof of concept they created.
The world of AI LLMs, and chat-oriented programming, is no different. Anyone can ask for an app and get a reasonable prototype. However, I don’t think this means less programmers. While more people will build more PoCs or prototypes, that also means more people will need a professional to clean up their system and make it work, scale, and perform well. And probably help secure it better.
Programmers (developers, software engineers, etc.) will scale themselves with AI tech, as they’ll get the AI to do some scaffolding and initial work that they clean up. I’m sure the very best will build RAG AI systems that generate the type of code they want with a little training and input. I could see millions of AI assistants that help write basis code outlines, write tests, check for standards and code smells, and help the human shift left, catching small, simple, silly mistakes.
I think we’ll have more people producing software in the future, but I also think that those who know their industry well will be in more demand. They’ll recognize issues in AI-generated code, they’ll guide AIs better, and they’ll communicate more clearly with AIs. They’ll get their pick of the best jobs, and they might be better compensated.
Things they likely do today with their fellow humans.
Steve Jones
I listened to an interview with Grady Booch. If you’ve never heard of him, he has been a software engineer for a long time, developed UML, and worked at IBM and a number of other places. He has devoted his life to improving software engineering. He even told Bill Gates he didn’t want to be the Chief Software Architect at Microsoft.
At one point he had a less than flattering description of AI LLMs. Politely, they are unreliable narrators. Less politely, he feels that they allow us to build at a global scale, unreliable BS generators. This is because the LLMs that are stochastic parrots, which can produce some coherent results. Primarily they allow us to navigate a very large lake in space, i.e. the Internet. I think that’s true. I have found AIs to be pretty good search engines. Not perfect, but good.
Mr. Booch notes that LLMs, while interesting, are a shadow or a whisper of what humans can do. He’s critical of those who think AI is going to compete with humans. Here are limitations on what the tech can do, and Mr. Booch thinks that the approaches people are taking, architecturally, are wrong. As part of his work, he studies more of how humans work and think as a way of trying to build better software architecture.
Caveat, he does think we can build more intelligent systems, but it’s not with Gen AI/LLM architectures.
I tend to agree with him and do think that the LLMs are unreliable. They appear to be intelligent, but they are more predictive engines in many ways. They can be very helpful in many ways, but they aren’t necessarily replacing smart humans. They might help smart humans replace some other humans, but they are likely to be better assistants than replacements.
It’s a great interview and worth listening to. There’s a walk-through where Mr. Booch touched on computing as well as a very positive outlook on the future of his career and the things that he is working on. It’s long, (1.5 hours), but a fun listen. Give it a go and let me know what you think.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
redesis – n. a feeling of queasiness while offering someone advice, knowing they might well face a totally different set of constraints and capabilities, any of which might propel them to a wildly different outcome – which makes you wonder if all of your hard-earned wisdom is fundamentally nontransferable, like handing someone a gift card in your name that probably expired years ago.
These days, as I speak with others as a coach or just community peer, I feel some redesis about any sort of advice I give that isn’t tightly bound in some way to a situation. Any sort of general advice has me wondering if the things that worked for me will work for others.
I recognize that the way each of us see the world varies, depending on your situation. I think EVs are great and convenient, but I’m in a position where I can afford one and afford a charging station in my house. I don’t worry about the price of eggs or bacon, but that’s because I’m lucky. I don’t worry about health insurance because I’m healthy now.
The advice I give people about their careers is always tempered to remember my redesis. The things that worked for me might not work for others, so I’m careful to explain the situations where this worked for me rather than give the advice without context.
From the Dictionary of Obscure Sorrows
Redgate Monitor is growing to include more than just Microsoft SQL Server monitoring. We added PostgreSQL support in 2023 and that continues to grow. This post looks at a few changes we’ve added.
This is part of a series of posts on Redgate Monitor. Click to see the other posts
PostgreSQL MonitoringWhen we added PostgreSQL support to Redgate Monitor, I assumed that we could monitor most types of PostgreSQL installs. I was very wrong. Apparently everyone that’s forked or used PostgreSQL in different ways has changed how a DBA might need to monitor the system. At first we monitored on-premises installs only, but that’s grown.
AWSWe now support RDS. If you search for bluebox, you’ll see a PostgreSQL RDS instance. If you click in the card, you can see the workload, and more specifics on this RDS database.
We also support Aurora, if you use that version. Search for pizza and these cards appear, the right one being Aurora. This is an AWS Aurora database, based on PostgreSQL.
AzureWe also support Azure Flex PostgreSQL database. Search for the pgtips database and you’ll see this card.
If you’re in the cloud with AWS or Azure, we have some PostgreSQL monitoring available for you, with more coming all the time.
Give Redgate Monitor a TryRedgate Monitor continues to change and grow. Look at our demo system at monitor.red-gate.com and click the What’s New in the upper right, where you’ll see all the new features, like the PostgreSQL support changes.
You can also see the release timeline for more specific features in each version.
Redgate Monitor is a world class monitoring solution for your database estate. Download a trial today and see how it can help you manage your estate more efficiently.
For the last few years, we’ve seen no shortage of cloud migration stories and felt pressure from management who wanted to migrate our systems to the cloud. It seems that almost everyone I speak to has a story of having to move a system out of their owned or leased data center into a public cloud from some vendor. A lot of this is the movement of VMs from one place to another, which has me scratching my head. If we’re just running VMs, surely we can do this cheaper in our own data center.
Perhaps, though there are a lot of costs to setting up or running a data center, and it’s not easy getting a system in place that allows a bit of self-service for our customers. Especially while ensuring that images used are properly patched and secured, while ensuring lots of easy connectivity to storage that can be reconfigured easily. It might not be worth the effort for a few dozen VMs, but if you have hundreds of systems, maybe it is.
Maybe it’s happening. I keep seeing stories about repatriation from the cloud. I also caught the global data center trend report, which shows a lot of growth in the data center world. Vacancy rates are low and there is continued demand for building more data centers. Some of this is due to public cloud providers, some is from AI companies who need lots of power and GPUs, and some is from private companies looking to collocate their systems.
The world is becoming more and more dependent lots of servers in data centers. I expect that we will continue to see more data centers built, but I expect fewer and fewer private, corporate data centers. More than likely all of us will use someone else’s data center, even if we choose to own the computing systems. Even Basecamp, which left the cloud, is using a collocation facility for machines, which means they are using a facility owned by another organization and shared with other clients. However, they own the servers they use, which are just located in someone else’s data center.
If any of you have private data centers, my guess is most of those will slowly fade away over time. The cost of running them privately will exceed that of what vendors will charge. Data center vendors can spread the cost of buildings, power, networking, cooling, etc. across multiple clients, often hundreds or thousands. While you might not be in the cloud, and you may still own your own computers, you’ll likely store and connect them in someone else’s data center.
That means that most of us will need to be comfortable with limits on the hardware deployed and amount of upgrades available. In the cloud you’re limited to what vendors provide. In our own collation spaces, it might be what our core IT group makes available. I still expect database servers to be among the largest machines available, but there will still be limits to what most of us can provision. After all, most IT groups still want some standard configurations shared by most of their servers. That might be an interesting trade-off for some of us as the cloud might be more or less preferable in certain situations.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
There is a report that the lowest-paid employees at various banks are selling customer data, which is sad and also not unexpected. Those who have access to sensitive information can be tempted to part with it for a quick profit. I’ve seen this in the past in restaurants and retail stores. Sometimes with credit card or other data, but also with knowledge (or keys) that would let someone burglarize the premises.
There’s a great quote in the article that says: “The more employees there are inside a company with access to sensitive customer information, the higher the risk that access is going to be abused.” I think that’s very true in most companies, which is why governance and data controls are becoming more important in some organizations. I wish it were a global trend at all companies, but far too many don’t think there is a high risk, or that their employees will abuse their privileges.
I’d like to think most IT people wouldn’t be tempted to sell or disclose info, but I’m not sure that’s true. While many IT staffers are paid well, well is relative. Some might not think they’re compensated enough or they might feel pressure from their own financial stresses to compromise that data of others.
Privileged access is just that: privileged. While it can be a pain to have a separate account for certain access, the intention is that you are reminded that this account can access things that others should not see. In my career, I’ve tried to remember this and be careful with my access, though admittedly, I’ve done plenty of non-privileged things with a privileged account, being lucky enough that nothing bad happened.
Security continues to be a problem in so many places, and with the vast amounts of data we collect, and the growing desire for many organizations to let AI LLMs view that data, I suspect we’ll likely have more problems in the future not less. While we might not have as many humans that decide to sell our data, I suspect we’ll have many more AIs that can be tricked into disclosing it.
Steve Jones
I saw a post from Brent Ozar a couple of months ago promoting his Black Friday sale, which is really a Month-of-November-Training-Sale. I like that he does this, as I know it can take time to get purchases approved and not many techies want to spend a few thousand USD in hopes their company will reimburse them. Redgate likely would for me, but I don’t know many other employers that would reimburse the expense without prior approval.
In the post, Brent outlines how you might spend your year working through his courses, learning more about various aspects of SQL Server. He plans out 11 months (Dec-Oct), and it’s a good flow going from understanding how things work to more advanced work in different areas. I’ve been in a few of his classes and I honestly think you might easily need a month if not 2-3 months to get through the various modules, practice using them, and cement in a few skills along the way.
That’s around the rest of the things in your life. You do need to live your life, right? It’s not just work and training for work?
I used to make a plan every Dec/Jan for the year, with some goals for learning and growing. I gave up a few years ago because I found the nature of my job meant my plan was likely not appropriate after March. Even going to quarterly goals was hard, as travel became a bit chaotic for me.
As much as I like to set an example, for someone in their 50s with a busy job, the reality is I don’t know where my job takes me year to year, so planning is really hard. Plus, I find myself focusing on different aspects of technology or software sales as challenges arise. While I’m trying to work on my career, it’s more tactical in the sense that I worry about new things in Redgate month to month that can help me in my position.
However, for many of you, you have a job that you know the parameters of, you know what types of things you’ll be asked to do, and more importantly, you know which of those things you were asked to do in the past, but struggled to get done at an expert level.
In that case, have you made a learning plan? Are you setting aside time most (or every) week to further your career? If you’re young and single or married without kids, you definitely should spend time every week on your career. Doctors, lawyers, CPAs, and engineers have dedicated education requirements in their jobs, and they spend time on their careers. You ought to aim for at least a 3-4 hour a week commitment to yourself.
If you have young kids, time is precious. While you might take a few years off, my view was that when my kids were 4-5, I could take some time to grow my career. In the those toddler/young schoolage years, this might only be an hour or two a week, but it should be something. As your kids grow, you can spend a bit more time.
If you’ve got older kids, now is the time when you can prove you’re worth a higher salary. Dive deep into tech and show that you’re an expert. If your current employer doesn’t want to pay you, I’m sure someone will. While there are lots of people looking for jobs, and lots of unfilled positions, employers aren’t just hiring unless they are convinced someone really adds value. Learn to do this and you’ll have a fun, enjoyable twilight to your career.
Think about your learning plan for 2025 and if you want to share, we’d love to hear it.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Experience is overrated. Most breakthrough accomplishments were done by people doing them for the first time. Therefore when hiring hire for aptitude and attitude and then train for skills. – from Excellent Advice for Living
I like this advice. It’s what I’ve tended to do when I have been a hiring manager. I look for people that have potential, can learn, and fit with the team. Not necessarily think or are the same, but they will grow with us. I think aptitude and attitude matter more than experience often.
There are exceptions. Sometimes we need skills and help in a certain area, and experience can be important. In those cases, I need someone that can teach others and is willing to do so. I still want aptitude and attitude, because even an experienced person will need to grow and learn (and get along).
The other trait I find important is curiosity. I need someone that wants to learn more.
I’ve been posting New Words on Fridays from a book I was reading, however, a friend thought they were a little depressing. They should be as they are obscure sorrows. I like them because they make me think.
To counter-balance those, I’m adding in thoughts on advice, mostly from Kevin Kelley’s book. You can read all these posts under the advice tag.
I’m back today from holiday, but gone tomorrow. As I try to close down the year, I decided to re-run How Much AI Do We Need? as it was an interesting look back at a piece from 2020, when the LLM/GenAI hype wasn’t a thought in most people’s minds. Instead, the AI systems were simpler and more focused on specific siutations.
I find myself thinking back a bit and thinking about the world back then and re-reading the linked article. In that one it seems like “smart” is more about better communication and coordination in the kitchen, and less about the appliances having any intelligence. Some of the image recognition items might lightly be called AI/ML work, but most of the added features aren’t.
Re-read the piece and article and think about how much AI do you need, or want.
As we close out the year, I decided to drop this post here and maybe inspire a few of you to write in 2025. This post looks at some examples of how I capture ideas for later and jot notes that turn into these blog posts.
Everything is an ideaLet me start with how I look at the world. I was in a meeting with some of our solution engineers and Kathi Kellenberger. Kathi was showing how to get started with the Redgate Monitor PowerShell API, which is something relatively few of our customers use. However, it’s very powerful and as she started, I noticed a few things.
- She pointed out how to download and load the module
- She showed how to make a basic connection
- There was a section on adding tags to a list of servers
- There was a section on reading in a CSV with server names and settings
- There was a section on changing settings for alerts based on the csv
In this list, there are 4 blog posts at least. Here is what sketched out. Each of these sections is the draft title of a blog post with the items I captured inside.
Getting Startedpurpose of PoSH with RGM
download module
get an auth token
make basic connection with your monitor server
Get data from a query
Adding Tags to a Servercheck sample scripts
Pick group of servers, get group list from
load csv, loop, applying tag to group
Load CSVBasic PoSh
Load CSV, show how fields are represented
loop through different rows
Change Settings from PoShGet list of servers and settings in csv
load and loop through
alter alert setting for server based on csv
SummaryThose are light sketches, but they capture the idea. I’ve got 4 draft posts now that I can flesh out when I have time. While I was doing this, I grabbed a couple screenshots as well, which help me remember the context of what I saw.
This is the way I grab lots of ideas when they occur to me, without worrying about finishing these right now. From these descriptions, I can build a larger blog post, in this case, a series that I’ll add to my other Redgate Monitor posts.
Try grabbing ideas for tools, fixes, patches, changes, etc. during your workday. Drop them in a folder somewhere, use Open Live Writer, Word, whatever. Then when you’re looking to blog or improve you career or fix something, you’ll have a list.
lilo– n. a friendship that can lie dormant for years only to pick right back up instantly, as if you’d seen each other last week – which is al the more remarkable given that certain other people can make every lull in conversation feel like an eternity.
I’m very lucky that I get to travel and meet so many people, and that I have the opportunity to see many of them time and time again at different events. I’m truly blessed in this way.
I have lilo with many people, close friends, good friends, casual acquaintances, even new-I-just-met-you-for-five-minutes-once friends.
With many people, when I see them again, we can pick up a conversation or a familiarity right away. I don’t have a great memory for names, which is why you might see my look at your name badge, but I remember faces well. When I was a bartender, I knew many people by what they drank, but few names.
In the tech world, I pick up friendships once a year with some people, less often with others. A quick story.
I met Nate many years ago at the PASS Summit. I can’t remember if we met in the convention center or in the hotel up the hill, but we got to chatting and enjoyed a few minutes together. Since then, I’ve seen him at a number of Data Community Summits, and we always take time to stop and catch up and see how we’re doing.
I look forward to lilo every year, thinking I’ll get the chance to see Nate and get a hug, a picture, and a conversation. I’ve missed him the last couple of years, but was delighted to see him in 2024. I hope we both keep making the Summit or some other event until we both get to retire.
From the Dictionary of Obscure Sorrows
It’s the start of the holiday season for me. I’m off all this week and a few next week, so I’m really done for the year.
You get to re-read Choosing Sequences Over Identity.
A new feature added to Redgate Monitor Enterprise automatically. CIS compliance is something many enterprises think about as their auditors use this as a benchmark.
If you’ve never looked at the Center for Internet Security, you ought to glance at them, and check out the benchmarks they have for many systems.
This is part of a series of posts on Redgate Monitor. Click to see the other posts
The CIS SQL Server BenchmarkYou might get asked by an auditor how you know your SQL Server estate is secure. There are lots of things you can do, but an easy one is being CIS compliant. There are benchmarks from CIS for many SQL Server versions. You can download the benchmark from CIS as a PDF, go through it, and then start to compare that to your SQL Server instances.
That’s not complex, but it is complicated. Lots of moving parts, where do you keep the benchmark data, how do you compare it to your instances, how do you ensure it’s up to date or get notified if it’s not?
This is a simple job, but labor intensive, boring, and tedious. There’s a better way.
Redgate Monitor ComplianceWe’ve added a compliance section to Redgate Monitor, which I’ve written about in terms of looking for older versions. However, we also have added to this section with a CIS Benchmark template.
At the top of Redgate Monitor, there is a Security section and Compliance is under this.
When I get to the compliance screen, on the right side, I have a drop down for the templates. We’ve pre-loaded the CIS Benchmark in here. I can select that to see how compliant I am.
In this case, I’ve filtered to the SSC servers and when I do that, I see that I’m mostly compliant, but just barely. I say me, but this is our IT group that manages the config.
There is a disclaimed at the top, which you should note. It links here, where the docs note that this is a template that cannot be deleted or changed. It can be duplicated. Note, this is only for SQL Server 2022.
If I click a server, I see the details of where and where not I am compliant. In tis case, things like database mail ought to be disabled.
I can’t change things from here, but I can export this as a report and work on remediation. If I want to set a template that is like CIS, but I have a good reason for an exception, such as the Cost threshold for parallelism set to something different, I can duplicate this template and alter it.
SummaryAuditing and compliance are becoming more important at many organizations, especially in light of the main data breaches and other issues that many organizations have experienced. This might even be required by insurance companies who want to ensure that you have not left open configurations that might become attack vectors.
If you haven’t tried the compliance templates in Redgate Monitor, give it a try, or have a play at monitor.red-gate.com..
Redgate Monitor is a world class monitoring solution for your database estate. Download a trial today and see how it can help you manage your estate more efficiently.
Microsoft constantly releases new features and products in the data platform space. Many of us have seen the SQL Server product grow in new ways, some of which are very useful to us. As an example the changes from log shipping to clustering to Availability Groups has improved our HA/DR options as well as the capabilities available to us in different situations.
With that in mind, I saw someone recently that wanted to deploy SQL Server on Kubernetes, which is something that could be a very interesting way of managing your different systems. However, this individual wanted to know when Microsoft would release their own supported solution with a Microsoft operator to manage the instance. There is guidance from Microsoft, but no official operator.
I saw a recommendation to use DH2i, which has a solution with an operator that can help here. In fact, in the MS docs, there are articles on using DH2i with SQL Server. However, the docs note that DH2i is responsible for supporting their product. The person asking about support didn’t want another vendor and wanted a Microsoft solution.
Is that something you want? Do you want a Microsoft solution for most (or all) things? Or do you think third parties or bespoke solutions are acceptable? This could be your opinion or a policy/guideline from your employer, but let us know.
To me, I think third parties are necessary. Microsoft can’t do everything, and they might not provide the support or flexibility that someone else can. Many of us sp_whoisactive, which isn’t an official Microsoft solution. There is a First Responder kit, diagnostic queries, a pressure detector, and plenty of other resources that people have created and shared. There are plenty of tools for SQL Server (and most other products) that various vendors have produced and sell which meet the needs of their customers.
My view isn’t to choose a Microsoft solution for everything because their solutions aren’t always the best choice for my problem. Even when they work well, they are often incomplete and I need to do some work to get them to fit into my environment. To be fair, most anything often needs a little work (or time) to fit into many environments.
Let us know today. Do you want official Microsoft solutions for your data platform? Or Oracle ones for Oracle? Who makes you comfortable with PostgreSQL, Aurora, Redshift, Databricks, or other platforms? Or do you only use tools and capabilities inside the platforms?
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
ochisia – n. the fear that the role you once occupied in someone’s life could be refilled without a second thought, which makes you wish that every breakup would include a severance package, a non-complete clause, and some sort of romantic placement program.
I struggled with this one. I have had quite a few friends in the last few years that got divorced and my heart goes out to them. Some ended up in financial hardships, so oshisia seems like a poorly-timed joke.
However, I get the idea. I’ve been attracted to someone, enjoying the dating, and then had it end. I’ve also ended it. I get that there is not only the loss, but the anger, the sadness, the wandering, the fear that someone else can easily replace you.
I think many teenagers who’ve been in love go through ochisia at some point.
From the Dictionary of Obscure Sorrows
I had someone ask me how to rename a SQL Server database recently. They were doing some development work and wanted to rename databases to test an application. I thought I remembered, but in this post, I show I learned something.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
Using sp_renameI thought sp_rename would work, and sure enough, it did.
However, I need to object type. If I remove that parameter, it fails:
The command is looking for an object in the current database by default.
Technically, I ought to do this to be explicit, naming the parameters.
I have a better way, however. Note: it’s not sp_renamedb, which is marked for deprecation.
ALTER DATABASEI don’t know when this changed, or if, but you can use ALTER DATABASE to change the database name. There is a MODIFY NAME option for this command that works well. You can see this below.
This is very clear and seems like better DDL For this process, which can easily be captured as code without worrying about parameters or ordering or anything else. I’d recommend using this.
SQL New BloggerThis post took me about 10 minutes to write. Easy. I had done a few experiments and I had code ready (which went to the customer), so I didn’t spend time there. Just rewrote what I did and learned in a few minutes.
You could do this, add to your blog, and maybe get an interviewer to ask you about this after they saw your post.
Recently I was watching a presentation on how to scale performance in your SQL Server environment and one of the suggestions was setting up Availability Groups (AGs) and having read-intent connections that would query the secondary and not the primary. It’s not a bad idea, and the SQL Native Client (and other drivers) support this and make it easy to implement.
The pattern of using multiple connections in an application, one for reads and one for writes, has been suggested often. However, in practice, I’ve rarely seen this work. Apparently having a connection variable, named dbConn, for writes and a second one, named dbConnReadOnly, for reads is too complex for most developers or teams.
Or maybe the idea of having to pick the right access point is a human problem? I’ve seen no shortage of problems in restaurants when we have specific “in” and “out” doors. Lots of people go through the wrong one and we end up with plates and food on the floor. Even broken noses or fingers at times. Perhaps I shouldn’t pick on software developers too much.
How many of you use two connections from apps? Meaning, do you think about reads and writes in separate connections. Even if you read and write from the same database, this can be a nice practice that future proofs code. It’s a small change, but it gives you room to grow if you get a read replica for analytics or reporting.
Of course, you could take it too far with different connections for different “services”, aiming for a microservice-style architecture. We could have dbUser for user stuff and dbOrders for the business side, and other connections for other services. I wouldn’t do that, as I think many of us will get confused, and we’ll often be doing two different type of service things in the same code. If I need something from a customer to write an order, do I have two connections in my method? I could, but I bet lots of developers would try to re-use a single one.
Plus, if developers get into trouble with two connections, then what will they do with 5 or more? There are lots of ORMs that might even support this, or if they do, not make this easy to code.
I’ve always liked the idea of separating reads and writes, but maybe the better solution is using one connection whenever we have simple CRUD work and another one for any sort of complex querying or reporting. That would make more sense as I suspect many of us will eventually offload reporting or analytics in some way to another system. A Delta Lakehouse of some sort seems likely if the current trend takes hold in more organizations.
Let me know today if you used (or have tried) different connections for reads and writes.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
The episode on data masking and subetting is out. You can see it here:
- Youtube
- Spotify
Watch and check this out. This is especially close to my heart as I’ve been pushing for subsetting from Redgate for a long time. I think subsetting is incredibly important for development agility.
Some interesting thoughts on the problem space, which isn’t a simple thing to solve. It’s not that complex (hard to understand), but it is complicated (lots of moving parts). As I’ve talked with others and worked on the problem, it’s not something that you can just knock out quickly.
What’s funny to me is that Andy Warren and I asked Redgate for this in the early 2000s and they didn’t build it.
We have a subsetter and masker at Redgate in our Protect/TDM area. Check those out if you need a solution for your org.
One of the things that I’ve been asked in every operations situation is what licenses do we need for our servers. This is a rare request, often once a year, but it results in a bunch of work to figure out what’s running.
I hate these requests because they always cause delays in other work.
This is part of a series of posts on Redgate Monitor. Click to see the other posts
What’s Running?Redgate Monitor makes this easy to track (if you’re monitoring all your servers). In the Estate tab, there is a Licensing item. If you check this out, you see something like this:This is a quick view of our systems, letting me know what cores we have in service based on the edition. This is a quick report, and if you’re monitoring all prod systems, this gives you a pretty close look at what you need.
The exceptions here are that we are reporting all cores for all nodes in HA/DR setups, and that might be more licenses than you need. However, you can subtract all the standby nodes if you have Software Assurance on those systems since you do not need to license those.
Of course, you shouldn’t need to license dev/test systems if you run developer edition, and you should.
You can filter by groups or other items that can help you focus on part of your estate. This is the standard Redgate Monitor at the top set of filters.
There also is an “Export” button on this page, so you can save off your data as an Excel XLSX.
SummaryThis section of the Estate tab gives you a quick view of your obligations to Microsoft for licensing. It’s a nice way to track this and report on it when you need to audit this information without spending a lot of time compiling the information.
Your Finance group will thank you.
Redgate Monitor is a world class monitoring solution for your database estate. Download a trial today and see how it can help you manage your estate more efficiently.
I assume most of you work with others in a team. Even if you are the data specialist and others work on different technologies, you still have a team. How long has your current team been together in this form? Have you had a stable team that might have grown, but the rest of the individuals and roles/responsibilities stay the same? Or has your team changed makeup, roles, responsibilities, or something else?
I don’t see a lot of organizations that change their team structures often. There may be people who come and go from a team, but the core structure remains the same. Even when your company might reorganize a bit, often it’s teams that shuffle between managers, but mostly remain the same. There certainly are exceptions, and some large orgs (Microsoft, Amazon, etc.) regularly shuffle lots of people around, but I’m not sure the teams change their makeup or their mandate much.
I was thinking about this as I read an article on knowing when to restructure your team. I won’t recommend you read it as I think seems to imply restructuring technology teams will make them perform better and start meeting all the commitments that have been made. While I do think that a well-led team can perform better, restructuring your teams isn’t likely to make them more efficient and productive. That being said, I do think the article raises some good questions about how you might evaluate your team.
There are certainly times when an IT team, whether in development or operations, might start to miss deadlines or may seem to work inefficiently from the outside. We are human, and humans can get complacent, or they might focus on tasks or work that they want to complete, ignoring work they don’t enjoy. The latter might be things the business needs, and restructuring the team isn’t going to fix that. Either the current staff has to be managed more closely to get them to focus on necessary work, or maybe different people should be assigned to those projects or tasks.
Just remember you work with humans, and they often struggle with change. Change might be necessary, but a little empathy helps us cope with the challenges and learn to work together in our new structure. If we don’t have that, likely nothing gets better and we fall into old habits.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
It’s been an amazing week here, as well as a long week. I’m tired, as are many others. The blogging table is a little bare this am, but I do understand. Louis Davidson invited me to the Simple Talk breakfast and it was a struggle to get up, pack, and make it to the event. I was a little late, but I got there in time to catch Louis’ remarks.
This morning the keynote opens with Ben Weissman, who gave last year’s community keynote with Melody Zacharias.
I love that we have the previous year’s speaker introduce the next one. Ben tells us that he started growing his career here at the Summit, as have many others. Ben tries to help others grow and expand their careers, and I hope a few people get inspired to start blogging, speaking, and more. Most of us you see on stage started just like those of you in the audience.
This is live-ish blogging, so apologies for typos. I’ve tried to italicize my thoughts.
PrivilegeWe are privileged to be here. Our companies invested in us, or we can afford it. We get the time to come. We get to interact with others in our industry, get some swag, enjoy the
Ben thanks the DEIB committee, which does a lot of small things. The code of conduct, the non-alcoholic bar, gender neutral bathrooms and more. These items might not affect you, but they make the event better for others.
There is also a New Stars track for new people who have never spoken. You can submit to this, which is a separate process from the main submissions. Ben runs New Stars of Data (with William Durkin) and he can help. He’s more than willing to help.
He also thanks a lot of people in the community that helped him. I feel the same way, in that we work hard, but success comes with help from others.
You might think others are doing much better than you. There certainly are those people, but there are also many more doing less well than you.
Appreciate what you have and acknowledge your privilege. You don’t need to apologize for it, but don’t pretend it doesn’t exist.
Generating InnovationGinger comes out, asking about the media we used to watch. Gilligan’s Island for her, on a TV. Today, kids watch YouTube, Tiktok, etc., and on their phones. They watch things like this.
Cable TV is going away, and a survey of younger people showed most thought this was going to happen.
Today everyone talks about AI and that we might lose our jobs. Ginger thinks this is hype.
But it is an important tool. Learn, innovate, be better.
Netflix is the first example. They worked by mail, competing with Blockbuster. They took a chance on streaming, which I remember. Wondering if we could actually get enough bandwidth. Today Netflix dominated over others.
First is not the most successful. Lots of tech was first, but hasn’t survived. Visicalc, Lotus 1-2-3 were first and everywhere, but they didn’t survive. Excel took over. They had a button to emulate Lotus, so that users could move. Excel innovated and won.
You need to innovate and change to survive. Ben is the example. He was (is?) a VB6 developer. However, if he stuck there, he might not be on stage, in the community, owning a company, and more. He changed and innovated.
That can be each of us. Ginger reminds us that other people help us grow, learn, get inspired, and do more. People are the reason companies innovate.
Innovation requires curiosity, brave, and willing to fail. You have to take risks. If you start something new, you won’t be great, but you can learn and finds ways to innovate.
SpaceX and musk shown. Why do some companies succeed in new ways? I think this is a great example, as this company changed the way rockets and space exploration works.
Who remembers and awful job. Ginger’s gave her headaches every Sunday, in anticipation of going back to work Monday. She didn’t plan this, but found herself in the situation.
We need to prepare for the future, be ready in advance. Explore your world, look at what’s important, what are others doing?
I hear this a lot from people, but the devil is in the details, and I know that lots of people question what to do. I both agree with Ginger’s advice, and I don’t. A few years ago we were told we needed R. Lots of people, including me, spent time on R as it was added to SQL Server, ML was a big deal, and we saw others learning this.
Today R isn’t something I see widely used. Python has taken over more, Databricks and Spark and other tech is being used more often, but will that continue? I don’t know.
Ginger worked at a company she thought was good, but started to see people leaving because they weren’t valued and didn’t work on interesting things. Money wasn’t the reason, and I agree. I think back to Drive, which talks about Autonomy, Mastery, and Purpose. Monday matters, but those things bring more satisfaction (if the money is good enough).
Ginger overworked herself at a job. She got sick, but thought she’d have done a good job. She asked why she was asked to do all the work in December.
“did you get it done?”
“She said yes, and the response was, “I guess I picked the right person for the job.”
The boss felt he’d done a good job, but Ginger didn’t feel that way. She felt like she’d been abused and taken advantage of in the situation. She wasn’t valued. The same feeling of her friends.
She left and that company failed. That company didn’t value employees. That could be us? Do we value our situations, our friends, our knowledge?
I think that’s a decent analogy, because there are companies (that I know) and people who do continue to survive and do well even though they aren’t liked, or good, or anything. Sometimes they take advantage of the market (companies) or their situation/power (people) and others don’t like them but feel trapped. Or they constantly find new people that don’t know how toxic or unsatisfying it is around those people.
I don’t know if those are exceptions, or if they’re the minority or even majority. I do know that those aren’t places I want to work, or people I want to spend time around. I also know there are lots of other opportunities.
Ginger mentions pay, and that more pay doesn’t necessarily ensure you’ll enjoy the situation. I’ve learned that the hard way as well. I have often found that more pay isn’t better. I can get better pay and a better situation, but I can also accept slightly less pay and enjoy the situation more. Have I done this? Yes. I made $75k at one point and took a $68k job. It was totally work it.
The Mississippi Miracle. After being ranked #50 in a 4th grade reading assessment (9-10yr olds), the US state found a new way to teach kids to read. They trained teachers, made an investment that was small, and they improved dramatically.
I think this is a great example. A little investment in your career is what I ask. Not a part-time job investment, but a 1, 2, 4 hour a week investment. It can make a difference. I was talking to someone about blogging last night. I said you could write 1 small post a week. In a year, that’s 50 posts. That’s a lot of stuff to showcase about your career, a lot of practice to learn better communication skills, and a lot to learn about what you know or do. If you want a better year in a year or two, start now.
Netflix had a recession, they laid off 30% of the company. They realized they had good people and had to let those people do work. They trusted them (autonomy), they asked them to work on things that helped the company (purpose), and gave them opportunities to showcase their knowledge and improve (mastery).
If you haven’t read about things the Netflix engineers do, or seen talks from them, they have done amazing things. They innovate in tech. Their engineering blog is amazing, even though I don’t know most of the tech behind what they do. I’m frequently lost, but I also recognize these are very smart people getting things done.
I’m not saying Netflix is amazing culturally, but they are impressive.
Ginger notes she looked for opportunities and tried to find ways to get into a new department. They kept promising things, but never moved her, so she quit.
Glassdoor, mentioning this is a good resource. One review, best thing is the food trucks outside because everything inside is garbage.
Take Glassdoor with a grain of salt. There are always people who just gripe or are unhappy. My view is toss away 5% of the top and bottom reviews.
It can be helpful in determining if you can trust the company and management.
Training provides opportunity. If your employer provides them, great. If not, you need to do your own training. You want to create your own safety net for the future.
I believe in this. In places resistant to investing, I usually do some, show what I’ve done and then ask for matching. That has worked in multiple situations.
Ginger notes that she talked to someone who got upgraded on a flight. They talked to a manager/owner next to them. The owner noted their employees were in the back and that’s a bad message.
Yes and no. Costs are a factor, and sometimes people get benefits because they travel a lot. I get upgraded and don’t apologize. I’ve had bosses that were in first while I was in coach because they had status and I didn’t. Or they spent money on an upgrade. I’ve also had bosses that use the same policy as employees. We do that at Redgate.
Don’t get too black and white with specifics, look for trends. Fair and equal doesn’t always mean the same for everyone.
What’s important to you? Talk to people to find out. Learn what matters to you. If you don’t like your situation, find a better situation and quit.
It costs 150% of a person’s salary to replace them. Is that crazy? Think about all the time, all the effort, the work not being done, the training time. Ginger thinks this isn’t necessarily a bad number.
Lat year a company gook their employees out to dinner at an expensive dinner at PASS. They then told the people because the food is expensive, they’d order family style and share. Made the employees think that management didn’t value them.
I’d agree here. Don’t cheap out if you make the decision to spend money.
Example of an employee that lives in S Dakota. The employee sends this person to a conference in Florida in December as a perk. He asks, chooses a time and place that suits him, and is happy. He’s still at the company.
Good employees get more opportunities and more perks. It’s fair because they often provide more value to the company. Keep that in mind when you ask for something and it gets declined. Some managers are just jerks, but I often question myself and sometimes realize that I might not deserve something.
Prompt EngineeringEveryone needs to learn how to do this. Take a course, and it doesn’t take 6 hours. Do these things:
- be specific about how much text in the answer (give me 5 things)
- pick the persona you want to answer the questions (act as a travel agent)
- ask for clarification from the previous question (give me more)
- ask AI how to ask AI a question (how could I have asked this better)
You can learn more by doing this. With generative AI, which generates responses, but also generates hallucinations. Because of this, ask for links and check them.
Ginger showed her Dad how to follow this method with AI and he was surprised by his success.
Go out and do good prompt engineering. This is a tool, and Ginger is more productive. It doesn’t do all her work, but it does help her. ZeroGPT is useful to find AI. Remind your kids about this.
I use this and others. I don’t take AI stuff at SSC.
Use tools, but remember we work with people and want to maintain our network. Remember the two rules:
- The Golden Rule – treat others as you want to be treated.
- Wheaton’s Law – Don’t be a jerk
Most of us can imagine someone in our careers that didn’t obey Wheaton’s Law.
I think about both of these often.
Ginger reminds us that don’t just live in AI, remember to talk and work with people.
Spend time with people who do what you want to do, learn from them, get inspired, and innovate for yourselves.
Recently I had a friend traveling who is not very tech savvy. This person has traveled before and has a routine, but in this case, they were struggling to get an airline’s mobile app to work. They also struggled with the website, and just before the trip, they were thinking to cancel because they didn’t have a ticket in their hand before driving to the airport.
This turned out to be a login issue, and between friends and the airline’s customer service, they were able to print out a ticket at home and take it in hand to the airport.
It was slightly funny to me, but it got me thinking about the fears people have about technology. Many of these fears are really about change, but I would guess most of us have been burned by technology at some point. There’s an app, a process, a system that fails and causes us stress/hassles/annoyances/money/etc.
Today I’m wondering if there is a technology are you afraid of or that you worry about more than average. Perhaps you have friends that worry about technology and have concerns. Drop a note in the comments, and I bet we’ll have some funny discussions.
This summer, my wife and I took a ride in a Waymo in San Francisco. It was a neat experience, and I enjoyed it, but when my wife posted a video, we were amazed how many friends of ours were unwilling to consider riding in one. We are Tesla owners, and we’ve been impressed with FSD, but my wife is also unwilling to use the basic Autopilot most of the time because it’s not reliable. Even FSD was something we were careful in using.
I live in the digital world often. I rarely use paper for anything, other than taking notes. Even then I often take pictures of notes and upload them for reference or for retyping. I pay for most things with digital money. I walk through life where technology manages supply chains, border admissions, airline flights, and almost everything I interact with. Even the vehicles I use at the ranch have lots of technology in them, though I admit I’m glad when something has analog parts I can work with.
I’m not sure there’s much technology I’m afraid of, though I am wary of some connected devices. Not because of the tech, but because I don’t think the vendors do a good job of security.
I can’t really think of what technology worries me. Even AI is likely to be less dangerous/useful/amazing/whatever than people think. I think the one part of tech that worries me the most is how it affects society and many people whose jobs are threatened by tech. Though maybe I shouldn’t worry. Those tech-averse people might be the reason we see growth in the travel agent world, which I thought was dead.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
skidding – v. intr. the practice of making offhand comments that sound sarcastic but are actually sincere and deeply felt.
I used to make lots of sarcastic comments. Some got laughs, some were appropriate, but most were a bad look for me and didn’t help any situation. Unless they were intentionally funny, and even then, I’m only lightly funny.
I’ve tried to avoid skidding with anyone these days, though I know I still do that. Rather than being sarcastic, I try to actually be sincere and heart-felt. I think it’s a better way to live life.
From the Dictionary of Obscure Sorrows
I missed blogging yesterday as I was on stage/backstage for quite a bit of the keynote. Live updates, so keep refreshing.
Today I’m blogging and noting the interesting things I see. Today is the Redgate keynote, and disclosure, I work for Redgate, so take this with a grain of salt.
The opening this morning is from my good friend, Annabel Bradford. Annabel and I have been at events all over the world for over a decade, both Redgate hosted and third-party events, including many PASS Summits.
She’s welcoming everyone, sharing some memories from her previous 14 events, and giving a little housekeeping. It’s good to have a few reminders as there is so much happening. The
Annabel also reminds us to give feedback. Fill our evals and surveys, take pictures and post notes, and even email the Summit team if you want to see changes. As someone that has been on the back side of these events, we review and debate lots of feedback.
The Redgate Keynote: Simplifying Complexity: making the database work in the real worldThe first speaker is Kellyn Gorman. She’s been a friend for around 15 years, formerly living in Denver. She’s come to SQL Saturdays and I’ve spoken at Oracle events, so we’ve crossed paths regularly.
The data world is scary, with so many data breaches. Lots of headlines.
And this is more painful because of multi-platforms. Lots of challenges above security, we have other issues, including just running systems
One of the possible solutions here is synthetic data, which shouldn’t be a security risk for data breaches. However, synth is hard, and Kellyn is leading to a new area where Redgate is spending some time.
It takes a village to get things running smoothly. She notes that AI and ML are here to help us, to be our assistants. She likes ML as a subset of AI. This will help us get more done in many ways, but one is with synth data.
Graham McMillan, CTO of RedgateGraham is our new CTO, I met him this past summer. We had dinner in NYC a few months back, and I’ve gotten to see how he thinks as there is a weekly CTO report that he post.
He used to run an SaaS operation with 5,000 SQL Server databases. They were upgraded semi-rarely, and the environment felt brittle. He has been in the situation many of us have, trying to get things done, testing our systems, upgrading, and feeling the pressure from the business. He believes in investing in people, process, and tooling.
He introduces an IDC, an analyst from Europe. Archana Venkatraman. I find analysts interesting in that they do hear from lots of different organizations and people, but they can still be led astray by a few loud voices.
A few stats from what IDC sees from executives
- digital tech spend expanding 7x faster than the economy in 2024
- CEO expect 49% of rev to come from digital products
- 8-10 expect more than 25% of new apps to be cloud native
Excellence comes from speed, quality, efficiency. We know that, but we also know that lots of organizations aren’t doing well in these areas.
Lots of tech debt, especially after the pandemic where companies pivoted quickly.
Graham doesn’t like tech debt as a term as those were the decisions we made. I think that’s a bit short sighted, as we sometimes make conscious decisions we know are wrong, but that need to be made to get quickly.
3.5 hours on average to deploy to databases. That’s crazy.
I also believe it.
Why, lots of silo’d development, especially between DBAs/developers. The DBA world is often working in less-automated ways, often from historical habits.
Speed is a bottleneck, which lowers our productivity. Less time is spent on value activities, and more on firefighting, That’s something I see often, and something many DBAs don’t appreciate.
The Messy MiddleMade some changes to address low hanging fruit, but then get stuck. This is the messy middle from IDC. The image shows this in two ways, with infrastructure shown, as well as operational challenges.
Costs is a big one that is pointed out, especially the lack of awareness and controlling costs. This is a big deal in the cloud. Visibility is another.
Graham says we’re here as we make a decision, but we don’t follow through. That is truly a leadership and culture failure, not a tech worker failure. Archana repeats this, culture is a problem.
How can we do that? That’s the trick. Redgate has some ideas, and we’ll see how that might be done.
Arneh Eskandari and Danny de Haan, come on. Arneh is a solutions engineer I’ve worked with for well over a decade. He and I have debated, argued, informed, and presented often in the past about DevOps and how to get better at working with databases. We often agree and I enjoy our talks. These days, we are trying to build better engineers to demo and support customers.
He’s her with Danny, who helps run the databases for the largest pension company in the Netherlands. Their approach to making changes is evolved. Five years ago, they worked very manually. Each team had to request all their own changes, knowing what they needed.
They have moved to building a platform to make things easier. Some are bespoke, built in-house, but there are also third party products. That’s likely the reality for a strong org, build some things, buy some things. Things can be done in a portal, or through an API, to request resources (servers, services, cloud, local, etc.).
This makes things better in these ways.
- easy (portal or API/code)
- independence (self-services)
- efficient, few people involved (self-service again)
- compliant by design (regulatory risk)
- secure by design (certain things allowed or disallowed)
Thoughtful automation – Arneh uses this term, which is what we should do. Don’t automate everything, and don’t just automate it for what the customer wants (or what the tech wants). Do it in a way that is compliant and secure by design, and give some choice, meaning API-first with a portal on top of this.
They have a baseline. They have an expected set of things that need to take place for infrastructure and code, meaning some limits/guidelines/restrictions. They flag things that deviate from the baseline. That’s an approach that I like, since there are reasons to sometimes go away from a standard.
However.
Most of the time the deviations are simple mistakes. We’re too far along in tech to allow simple mistakes to slip through often, which is likely in the era of constantly changing staffs. Use automation to flag these, but have a way to create exceptions if needed.
More Industry InsightsGraham and Kellyn come back.
“Devops is essential. If you’re not automating, you’re not doing a good job.” – Kellyn.
Graham notes we run the State of Database Landscape survey. We’re run this quite a few years, and this year (the 2025 report from 2024 research), we had 2500+ responses. Report coming in Jan, but a few early results.
What do we want. What’s efficient? Kellyn says update. She’s prefer a db with no users, but that’s not possible (laughs).
She wants to patch or upgrade the database platform, but Kellyn doesn’t want to do changes for deployments. She does like tuning.
74% using more than one platform, along with organizations trying to incorporate lots of new technology. 26% using more than 4 platforms. Audience survey, but I couldn’t see how many responded.
18% (almost 1 in 5) making daily changes. There is also a 50% increase in short term changes between 2022 and 2024. These are bug fixes, security fixes, or enhancements. I see pressure to do this, but I also worry about this. Short term can be rash changes if not thought through.
The mission of Redgate is to help here. We display this.
The
Redgate bought db-engines, which Kellyn has used for years. She shows growth across some top platforms, which is similar to what we showed last year. We’ve highlighted this at a few events this year.
Flyway supports databricks, Clickhouse and cassandra. MongoDb as well. N
Not just a breadth though, but we are adding depth. Lots of changes in Flyway, as well as Redgate Monitor, which now runs on Linux/Timescale (PostgreSQL) if you want.
We’re into the product listing.
TDM mentioned, which was announced last year, and they note we’re improved subsetting performance. I am happy to see this (I knew this as I’ve been helping the team lightly), because I think this is crucial for success.
Cloud MovementKellyn is showing the the all in the cloud numbers have gone down in 2024 from 2023. Why? There is an increase in all on-premises environments. This matches Kellyn’s experience as she has worked on a lot of VLDBs, which can’t work in the cloud. She’s seen some of the large systems struggle to work in the cloud and they’ve come back.
That’s not most of us, but it is some of us. I never thought VLDBs made sense for the cloud if they were heavy on compute and IO. If it’s just size and they can tolerate latency, great.
Initial investment in cloud movement is low, but if you don’t manage this, cost increases can get out of control. Another reason some orgs might be moving back.
Cloud marketplaces are the place for Redgate. We’ve added Flyway, Monitor and TDM in the cloud.
Biggest challenge from the audience? Data integration issues. The report, skillset requirements and training. However, this is an audience whose companies invested in them for the most part, so I think skills are less an issue here. They have more tactical challenges.
Back to products, with James Hemson. He helps run TDM and I talk with him weekly. The first item to look at is security. That has limited 21% of organizations to adopting other platforms. I guess that makes sense, though I don’t know new platforms are needed. Maybe.
However, that’s a business decision. We don’t want security to be the driving force unless the new db platform is insecure. In that case, don’t use it.
That brings us to Redgate Monitor Enterprise. We launched this earlier this year as a new tier. One of the things added in Ent version was compliance checking, where we look at configuration against a baseline. We also added some security information on access to servers and databases. James shows some of this.
I won’t, but you can see this online for permissions and compliance in our demo system. A nice quote: you can see you’re compliant all the time, not just at audit time.
People believe in AI. It improves productivity (at least a belief in our survey). Hmmm, not sure I think this, but I’m skeptical.
AI can help you in some ways. Redgate has been working to add AI to Monitor. Example, lots of alerts. Too many. If we lowered a threshold, we can remove a lot of noise. We can get alerts when things are very or unusually high, not just higher.
ML is being used to tailor alerts to limit them to when they’re important or needed, not just every alert. This is a great use, since many of us don’t need to keep tuning alerts. We need to respond to them.
I wasn’t aware of what Monitor was doing here, but I like it.
We have a lack of collaboration. Bringing Dev and Ops together, a la DevOps, should have solved this.
A bit of a stretch to me, but I was aware of this. We wanted to ensure we can get test data to devs easier, without a lot of Ops effort. We are adding beta synth data into TDM as of today.
I’ve seen this, and it’s got possibilities, but I’m still unsure of how effective this will be. You judge, but there are some interesting ideas with ML looking at your data and generating synth data that looks like your data, but it’s not real. Great idea, but we’ll see how well it works for lots of different data structures.
There are both rules-based and AI-based versions of this that can be used in different ways. It was an interesting idea when I saw it a few months ago, but I’ll leave it there. You test it and let me know if it works for you.
Monitor, automate, protect, the three areas where Redgate works. Depending on your job and your challenges, you might use one of these.
Was this keynote that showed how to simplify complexity in the real world? I think there were some good things here, though there are certainly some product focuses here on Redgate solutions, but the high level concepts are things that help us make the world better.
Not simple for all the people doing the work, especially in Operations, but for the users, life should be simple. Watch the replay
Final announcements, not just 1 Summit next November, but 4. Three more in other cities (NY, Chicago, Amsterdam), which will be smaller and shorter (1-2 days). I’ve known about these, but wonder if you like this.
Come to the Redgate booth and win an Aardvark, which represents the first product from Redgate. If you get the golden one, you get $250 as well.
Today is the first day at the PASS Data Community Summit and I’m in Seattle where I’ll get to open the conference and introduce the Microsoft keynote. I’m sure the keynote will be full of announcements on something, but what?
I’m writing this a little over a week before the event, and I have no idea what Microsoft will do. Actually, by the time you read this I may have some ideas as there was a practice session yesterday, but I can’t tell you anything. NDA, and really, by the time I got something organized, the keynote will be done.
You can stream the keynote, and watch the opening (and wave to me if you want) for the conference for free. Some of you may read this before that time, and if you do, then think about this question: what do you want to hear Microsoft announce or what would get you excited about data work.
I don’t often spend time thinking about what I might like, mostly because I can’t influence things and I don’t like to spend too much time dreaming about databases or what things would help sell more licenses or compute or anything else. However, I am interested to know if a new version of SQL Server is coming (and when) and what might be in it. Or if there are structural changes that might be interesting to me as someone who helps customers work with databases and data.
If you complain about the performance of SQL Server 2022 and want bugs fixed, that’s fine, but that’s not going to be in a keynote. Instead, think about what things could be added to SQL Server, Azure, Fabric, or some part of the data platform. Where will Microsoft go that might help our organizations or be interesting? Think about what you might want to adopt in a year or 2 if it works as designed.
I have no idea, though I suspect AI is going to be said once, maybe twice, or maybe even a few more times
Leave a guess in the comments. If you don’t watch it live, then take a guess and watch it tomorrow to see if you were right.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I’m already up. It’s just after 6am in Seattle, but I’m moving and getting ready to head over to the convention center. The keynote starts at 8, and I need to be there to kick off the conference and introduce the Microsoft speakers.
My day:
- 8:15am open the conference and keynote
- 10:25am – Essential Productivity Hacks with Grant
- 11am – Redgate lunch with the board
- 3p – customer meeting
- 6p – expo hall reception, where I hope to have a beverage
The rest of the conference has me in the community sessions, though not too many. I like that I’m not too committed as I was buried in 2022 and only slightly less in 2023. The rest of the week:
- Thur, 12p – Hobby sessions, come hear me talk volleyball
- Thur, 1p – Community conversations about user groups and events
- Fri 945a – Community leader meet and greet
- Fri 1145a – Emcee the Dev Lightning Talks
- Fri 4p – heading home, well the airport
This is a slightly compressed week for me as I really only arrived at the conference Tuesday afternoon. I spent a little time in Redmond Mon/Tue and then off Fri as I have Sat am coaching commitments at home.
If you are attending the Summit and see me, please say hi.
Traveling today to Seattle and running around to Redmond as well. No time to look at anything, so you get a republish: Idempotent
Enjoy and hope you learn something.
I can’t remember how I heard about Small Data SF 2024, but it caught my eye. The mix of sessions had me interested in going, especially with Mother Duck and Duckdb being the main sponsors. I’ve run into DuckDb a few times in the last couple of months, so I was interested in what I could learn about small data and a different group of people than I normally see at events.
When a customer visit cancelled, I requested the learning and development (L&D) time and budget and got it approved. I booked flights and a hotel and headed to San Francisco.
The structure of the conference was interesting to me. I’ve been lucky to get to a few smaller conferences in the past (100-200ppl) and I like them. SQL Bits, PASS Data Community Summit, and other events are nice, but I tend to like small events.
This conference had the tag line of Think small, develop locally, ship joyfully. There were other tags, and you can read their manifesto, but essentially this conference looked at the idea that lots of work with data (OLTP or analytics) can be done on small sets, with local databases or local data.
Day 1Day 1 started late, at 12p with lunch. I liked that, though I took advantage of the late start to sleep in and get a late breakfast, so I really wandered around and chatted with people with a coffee. Food was nice, as it always is in San Fran. Lots of dietary choices, and mixes of stuff. The event was in a co-working facility, so there were always snacks around (chips, nuts, fruit, etc.).
Day 1 was two workshops. Each was 3 hours with a break between them. Essentially these were vendor sessions for hands on work with a product. There was a happy hour after, but I skipped it.
My first workshop was from Mother Duck, a vendor building on DuckDB. The workshop was based on this github repo, and showed how to use dbt to move some data around. It was hands-on, and things worked well for me, but this was a mix of CLI work, python, database work, and more. Some people definitely struggled with the workshop. I found this interesting, and I learned a few things. I’m definitely interested in doing some DuckDB work to analyze data in a way that is different (and simpler) than Snowflake or Fabric. I could see people doing this.
The second workshop was from Outerbase, which essentially is a way to work with multiple databases on the web. It’s a light Object-Explorer/Query tool in some ways, but they’re also trying to do some AI work to help stub out a web interface for your database. They had us try to build some methods and web code that we could paste into a React or Angular (or others) framework. This one was OK, but I am not sure this is a great use of AI. I was hoping for a bit more.
Day 2Day 2 was all day, from 830-530. I arrived to find a lot of people getting breakfast. Again, hot food, cold food, GF, etc. Lots of choices. One cool thing was a coffee bar where you could get baristas to make a nice drink, but you could also get a Mother Duck mug for your drink. I have too many mugs, but I liked this one, so I got one.
This was a one track conference, which I also like. Everyone gets a shared experience, we have common things to talk about, and things change often. I also don’t have to go find rooms. In this case, one large rooms with a low stage.
New talks every 20 minutes, on a variety of topics. The agenda was wide and varied. I think a few talks were meh, but most were interesting. I’ve got some editorials coming, but the first talk on Big Data was great, as was the second one on different tooling we might use for both development and analysis of smaller sets of data.
Note, small data doesn’t mean kb or less. It notes that many queries can be run on GB of data on a laptop, and with today’s network and laptop capabilities, this can make sense. There was also some limited domain views of the ways you might shard your data to lots of databases, and you might do more local work, not central db connections. That makes sense in some cases, but not all.
I also think some of the speakers (quite a few startup people) minimize or don’t think about the true scale problems when workloads grow, nor about the hassles of pulling all this data together and synching it. In any case, I think their ideas work for some problem domains.
There were a few panels, as well as a presentation of a paper from Amazon. One super interesting thing was Redshift shows like a 60:40 split of reads to writes. That seems crazy. However, an exec from FiveTran talked about that matching their experience where many data warehouses are running constant updates from OLTP systems, something that their customers sometimes don’t realize. He wasn’t sure if this was a good idea as well, but it’s been good for their business.
As seems to be the case, there was a satirical talk on BI tools and how they don’t always help. An analyst for one of the political campaigns gave a funny humorous look at the world of vendors and customers.
After the last talk, there was a short happy hour, where I had the chance to chat with a few people. Silicon Valley is a strange place, full of people working in startups, formerly from startups, or wanting to start one. Everyone has a good idea, which I think is true, and so many of them want to chat about their thing or your thing.
As you might expect, a lot of people at AI-focused or thinking AI. It’s neat to hear their experiences and what they think. Certainly I saw some neat demos or using small models (again small data) and feeding a user query into the model along with some data from a database or a flat file. That was interesting and something I think could be useful in different ways that are focused. I expect more and more people to get comfortable with AI based work.
Ultimately, I had a nice, refreshing two days that got me thinking about data differently and how there are different ways to approach problems and solutions. Perhaps one of the neater things I saw was PySheets, Python in spreadsheets. Just don’t try it in Chrome, and make sure to use the little A* button to test the AI.
When is the last time you read an article/blog/etc. on the Internet and saw a button for a print friendly version? That used to be something on every page, and one people often shared on social media (or email) because it didn’t have all the advertisements in it. I remember having to help code this feature on SQL Server Central when we started as plenty of people wanted to print articles out and read them later. That desire led to Andy brainstorming that we should release The Best of books each year.
I was reading about how the Internet has changed many things in our lives and I thought about these links. I searched a number of places I visit often and there are no more printer links. I’m guessing with mobile devices and various save services, most people have gotten used to using digital technology to consume information?
I still print things at times, though fairly rarely. I don’t often consume anything on physical media anymore, including books. I’ve tried to read a few times on paper, but it’s inconvenient to me now. I have to remember to pack something or carry it, I need a light often, it just doesn’t work as well.
I rarely see paper in use in meetings anymore at all. Whether I’m at a Redgate office or a customer site, most people seem to have monitors, projectors, sharing apps, and more so paper is just rarely used.
At the same time, it’s not completely out of date. It works well and it’s simple. I see it used for announcements, for small handouts, signs, and menus. Quite a few of us don’t like the digital menus from QR codes and it seems most restaurants I’ve visited still create physical menus. Signs and announcements are the places I still see paper in use regularly. I will say I’ve seen a few people (a very few) using e-ink devices, which is something I’m tempted to use. I do find writing helps me remember things better.
The world continues to create more and more data, while finding more numerous and novel ways to disseminate it. For much of the time, the paperless office exists, and I see less and less use of paper for distributing information, but it hasn’t completely disappeared. Except, perhaps, from the web.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
moledro – n. a feeling of resonant connection with an author or artist you’ll never meet, who many have lived centuries ago and thousands of miles away but can still get inside your head and leave behind morsels of their experience, like the little piles of stones left by hikers to mark a hidden path through unfamiliar territory.
Moledro is something I get mostly from music. I’m less interested or thoughtful about most visual arts, though perhaps media counts. In any case, the poetry of some artists creates a connection with me that sticks with me.
A few examples:
- lately, Zach Bryan. his poems and songs connect with a younger version of my, full of hope, heartbreak, and silly youthful decisions.
- Bob Dylan, wanting a better world, and living a life that he enjoys.
- Prince, a man truly wanting to entertain and make music, but his own way. In many ways, we’re very different in how we approach life, but I feel this connection with someone that live his life as his career.
- Steven King – I wish I could write like him, or have the drive to do so, but I love his activism and opinions.
There are likely many more, but these stand out to me.
From the Dictionary of Obscure Sorrows
I don’t do a lot of work with disabled index, but I learned how to re-enable one today, which was a surprise to me. This short post covers how this works.
The ScenarioImagine that you have an index on a table. In my case, I created this index:
CREATE INDEX LoggerNCI ON dbo.Logger (LogID)
I can then disable this index with the following code:
ALTER INDEX LoggerNCI ON dbo.Logger DISABLE
I had assumed that ENABLE would be the opposite, but SQL Prompt taught me this wasn’t an option. I checked the docs, and sure enough, it’s not ENABLE.
It’s resume. This code turns the index back on and updates it.
ALTER INDEX LoggerCI ON dbo.Logger REBUILD
I can also use either of these items:
CREATE INDEX LoggerNCI ON dbo.Logger (logid) WITH DROP\_EXISTINGDBCC DBREINDEX(Logger, LoggerNCI)
Interesting short moment the other day as I realized there are a few options here.
SQL New BloggerWhile playing with this, I realized that I didn’t know all the ways this worked, so I spent 10 minutes after I’d worked with the code to put this together.
A nice short way to showcase some learning.
How many of you have objects in your database that aren’t being used? What about something in a schema with a _old in the name? Or _2 or _3 or _delete? There is a lot of old, deprecated stuff I see in production databases. In fact, I’ve been somewhat amazed as I work with clients that many of the scripts we can build from a database with SQL Compare won’t actually execute on an empty database because the script is full of broken code.
I also find plenty of DBAs that want to clean things up, but they don’t. Sometimes they’re afraid they’ll break something, which is certainly possible. Sometimes they can never find the time. Often they might ask a manager, who usually says this isn’t important and don’t bother.
Is it worth it to clean up your databases?
Brent says no for old code. I say maybe for tables and code.
For a lot of code, Brent is right, your boss doesn’t care and it doesn’t necessarily help you. After all, it’s in production now, and if it’s being used, you’re going to just create problems with a DROP. Where is the business value for removing old code (assuming it isn’t being used)? What benefits do your clients get? Not you being happier there are less objects, but what is the business benefit.
That’s the key. Is there a business benefit. What I’d say is that if you have broken code, it needs to be removed. Because this does impact your software development process, especially when trying to match lower environments. For broken stuff, save the code in your VCS (you do version control database code, right?) and then delete this stuff from prod. It’s broken.
Or fix it.
For tables, I would want to get rid of old tables as well. Why? Well, this is real costs in storage and potential reading of old data. If we moved data to table_old and someone decided they needed to read this for a report at the time, they might still be reading old data. I’d first rename these objects as object_delete_date with the date being a month away. Then I’d set a reminder for that date. On that date, bcp out the data, then drop the table. Period.
Two other things. First, make sure you know how to recreate the table (see the VCS comment above) and bcp in the data. Two, this is low-priority work. If you want to clean the database, know this is a long term, baby step process that will take months or years, and may never end.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I’m leaving again tomorrow for a trip. This time I head back to Boston for a Redgate DevOps in a Day on Thursday and SQL Saturday Boston 2024 on Saturday. This is a fun event and one I’ve attended the last few years. It’s one I look forward to as well.
If you’re in the area, register and join me Saturday for a packed schedule of learning on PoSh, Fabric, TempDB, AI, and more. If you want something a little lighter, I’m doing a new session on balancing life with career.
And if you’re coming Saturday, what about coming Friday for a pre-con with Bob Ward with a Cloud Workshop. A good, inexpensive way to get some training from one of the top technical engineers at Microsoft.
Come join me and invest in yourself and your career at SQL Saturday Boston 2024. And if you want, come Thursday as well to our Redgate Database DevOps in a Day.
I’m not the smartest developer or DBA. I find myself mystified at times by Itzik’s posts on T-SQL queries and I’m amazed at times by the complex systems that I see the DCAC people put together. I can usually figure things out (sometimes by asking the authors a question), but it’s not always easy to do. We have some truly gifted, incredibly intelligent people in this business.
I am, however, effective. I have been very successful in my career at getting things done well enough, things that work well, meet the needs of my client/employer, and meeting deadlines. I don’t just slap things together, but think about them, build them, test them (don’t forget this), and then make sure they’re working when they’re deployed.
Sometimes this might take a few PRs or patches for patches, but I get things done. And my customers/clients are happy.
I saw this post on Linked In noting Platform Engineering is Dead, which is a great title, but not really true, and not quite reflected in the piece. The author worked on the Software Delivery Enablement team, which is what the platform engineering team is supposed to do.
I see similar complaints about DevOps, and previously saw complaints about Cloud computing or Agile or Scrum or SOLID. There have been similar complaints about how some new methodology or idea isn’t working and should be abandoned in favor of this other new thing.
Ultimately, near the end of the piece on Linked In, there is this:“we also knew how to help them use solutions to deliver software better, and we partnered with them instead of inflicting things upon them.”
This is what Software Engineering should be. In waterfall, we want to have customers tell us what they want and build that. Often customers don’t know what they want, so we decided Agile would help. DevOps is a way of talking about a partnership between developers and operations that still delivers what the customer wants, quickly.
Platform Engineering or Software Delivery Enablement or whatever name you give it is still partnering with customers to deliver what they need. Not what you think they need or what you want to build, but what they need.
Whenever Agile or Scrum or DevOps or Platform Engineering doesn’t work, it’s because you’ve forgotten that this is a partnership. That’s what effective engineering is, and it’s what I’ve practiced. Partnering with others to achieve our aims.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
A customer recently wanted to know if any of their instances were too old and out of support. This was for a compliance purpose, and they had the need to show a report to management of when instances were out of compliance with policies.
This post shows how you can do this in Redgate Monitor.
This is part of a series of posts on Redgate Monitor. Click to see the other posts
Compliance TemplatesThis is an Enterprise feature, but it allows the administrators of Redgate Monitor to define templates for how the various servers and databases ought to be configured. Deviance from standards, or compliance with standards is often something auditors care about you documenting.
This is very easy with Redgate Monitor. Let’s see how this works.
Under the Security tab (appears in Enterprise edition), you can see there is a Configuration Compliance item. Click this.
This brings up the Configuration Compliance screen. This starts with Servers (highlighted with a bar to denote this tab, but there are also databases and Compliance templates. We want to choose compliance templates.
This tab shows the various templates that exist already in the system. I’m looking at monitor.red-gate.com, which has three templates configured already.
If you scroll to the right, we can see that two of these are server (instance) templates and one is a database template. We can add a new one, or edit existing ones. For this post, I’ll edit the Workload Server template.
After we click Edit, we get another set of tabs for the template itself. The default is for security, but there are also performance and environment options. Pic the Environment tab.
This brings up a list of settings. Scroll down to the bottom and you will see what settings below for Product Level, Product Version, Product version number, and Edition.
In this case, the template is checking that every server has at least SQL Server 2019 Enterprise installed. If any other version were installed, this would show as non compliant. This is useful for grouping your servers by versions.
The values entered here are the ones you would get back from the ServerProperty() function.
SummaryThis post shows how you can configure a compliance template to check that the versions of SQL Server you have installed meet your requirements for a version.
Redgate Monitor is a world class monitoring solution for your database estate. Download a trial today and see how it can help you manage your estate more efficiently.
Often I see running totals that are written in SQL using a variety of techniques. Many pieces of code were written in pre-2012 techniques, prior to window functions being introduced.
After SQL Server 2012, we had better ways to write a total. In this case, let’s see how much better. This is based on an article showing how you might convert code from the first query to the second. This is a performance analysis of the two techniques are different scales..
Pre SQL Server 2012The old way:
SELECT Acc.ID,CONVERT(varchar(50),TransactionDate,101) AS TransactionDate , Balance, isnull(RunningTotal,'') AS RunningTotal FROM Accounts Acc LEFT OUTER JOIN (SELECT ID,sum(Balance) AS RunningTotal FROM (SELECT A.ID AS ID,B.ID AS BID, B.Balance FROM Accounts A cross JOIN Accounts B WHERE B.ID BETWEEN A.ID-4 AND A.ID AND A.ID>4 )T GROUP BY ID ) Bal ON Acc.ID=Bal.ID
What were the statistics on this? After running a few times, with STATISTICS IO ON, I get this:
Table ‘Accounts’. Scan count 37, logical reads 37, physical reads 0
Not bad. I’ve truncated out the other values as they were all 0.
Window FunctionsHere is the same query written with a Window function.
SELECT
id
, TransactionDate
, Balance
, CASE WHEN LAG(TransactionDate, 4, null) OVER (ORDER BY TransactionDate) IS NOT NULL
THEN SUM (Balance) OVER (ORDER BY TransactionDate ROWS BETWEEN 4 PRECEDING AND CURRENT ROW)
ELSE 0
END AS runningotal
FROM dbo.accounts
The statistics?
Table ‘Worktable’. Scan count 0, logical reads 0, physical reads 0
Table ‘Accounts’. Scan count 1, logical reads 1, physical reads 0
The window function definitely does less work. A lot less. But how does this scale?
Performance TestingThere are numerous ways to create some test data for this. Since I have Redgate SQL Data Generator, I decided to use that. It’s simple and easy, and I added 100,000 rows first.
My results of the first query:
Lots of reads and scans. Let’s compare this to the window function.
Hmmm, both took essentially zero time less than a second. That might lead some developers to think either method is quick enough.
Let’s add 1mm more rows.
Now compare. The first takes about 15s with these results.
The window function? 3 sec, with these stats.
The comparison looks like this. First, let’s look at SSMS time
| Old, Cross Join | Window Function | | 20 rows | 0 sec | 0 sec | | 100,020 rows | 0 sec | 0 sec | | 1,100,020 rows | 15 sec | 3 sec |
If we look at CPU time, then we see this:
| Old, Cross Join | Window Function | | 20 rows | 0 ms | 0 ms | | 100,020 rows | 748 ms | 250 ms | | 1,100,020 rows | 5845 ms | 2919 ms |
If we look at the logical reads in total, we see this
| Old, Cross Join | Window Function | | 20 rows | 37 | 1 | | 100,020 rows | 403,998 | 359 | | 1,100,020 rows | 6,450,895 | 3945 |
Clearly the window function is better and the better grows as the size of data grows.
SummaryThis post looks at two queries and compares the performance across a few queries. These aren’t the only ones, and you might choose other types of queries, but these are both examples of how you might approach a problem using old tech and new tech.
The window function is not slightly more efficient, but extremely efficient compared to the older style method of using a cross join. As the data scales up, the difference is pronounced. While 1mm rows might not be a great test here, and you may prefer to test at 10mm or 100mm rows to get an idea of load, the fact is the Window function is much quicker and uses less resources.
If you are using older style code to perform T-SQL calculations, make some time to refactor that code (and test it) to use modern window functions.
Setup CodeHere’s the initial setup code:
CREATE TABLE Accounts(ID int IDENTITY(1,1),TransactionDate datetime,Balance float)goinsert into Accounts(TransactionDate,Balance) values ('1/1/2000',100)insert into Accounts(TransactionDate,Balance) values ('1/2/2000',101)insert into Accounts(TransactionDate,Balance) values ('1/3/2000',102)insert into Accounts(TransactionDate,Balance) values ('1/4/2000',103)insert into Accounts(TransactionDate,Balance) values ('1/5/2000',104)insert into Accounts(TransactionDate,Balance) values ('1/6/2000',105)insert into Accounts(TransactionDate,Balance) values ('1/7/2000',106)insert into Accounts(TransactionDate,Balance) values ('1/8/2000',107)insert into Accounts(TransactionDate,Balance) values ('1/9/2000',108)insert into Accounts(TransactionDate,Balance) values ('1/10/2000',109)insert into Accounts(TransactionDate,Balance) values ('1/11/2000',200)insert into Accounts(TransactionDate,Balance) values ('1/12/2000',201)insert into Accounts(TransactionDate,Balance) values ('1/13/2000',202)insert into Accounts(TransactionDate,Balance) values ('1/14/2000',203)insert into Accounts(TransactionDate,Balance) values ('1/15/2000',204)insert into Accounts(TransactionDate,Balance) values ('1/16/2000',205)insert into Accounts(TransactionDate,Balance) values ('1/17/2000',206)insert into Accounts(TransactionDate,Balance) values ('1/18/2000',207)insert into Accounts(TransactionDate,Balance) values ('1/19/2000',208)insert into Accounts(TransactionDate,Balance) values ('1/20/2000',209)go
The season 1, seventh episode of Simple Talks is out. Check it out, with Ryan as the main host.
Simple Talks is the Redgate podcast from myself, Grant, Ryan, and Louis. The main page is here, and it has links to the audio versions as well as a the video one.
Another one recorded in Austin, during the Redgate 25th Birthday week. We tried doing one on the roof, but it was too windy.
We ended up in the conference room.
A bit has changed in my AI view since then, with a second trip to San Francisco and people talking AI more and more. I’ll have a few thoughts later in another post on the Small Data SF conference.
I saw a note from someone recently that reminded me of Policy-Based Management. This was (is?) a technology in SQL Server that I thought might have great potential. I even had a few presentations on the subject, but sadly I’ve rarely seen anyone implement it. I’m sure some do, but I think for me, this is dead technology.
There have been other tech items from which I’ve been turned off or abandoned over the years. It seems in SQL Server, we have some tech that even Microsoft has abandoned and doesn’t put any development resources into improving.
This week, I’m asking if you have technology that’s dead to you. It could be in the SQL Server or Microsoft Data Platform, or it could be elsewhere. Let us know in the discussion below.
I have often moved on from some technology to something newer or better, but that’s not because the old software was dead to me, but rather because I found something better.
However, I have abandoned some things. Database Mirroring is something I see as dead, as does Microsoft. I think for me, any source control system other than Git is dead to me, though maybe that’s more because Git has proven to be better (and ubiquitous). I’d say that I find most multi-platform messaging clients dead to me, mostly because they can’t keep up or implement good interfaces that work across many places. At least not in a way that makes sense for me. I’d say these days, digital cameras (and video cameras) are dead to me as a casual snapper who can live with a phone.
I’m sure there are other dead technologies that you’re just annoyed with or think don’t (or can’t) meet your needs. There are likely many more which didn’t keep up with you over time, which you wouldn’t have abandoned if they had just improved a bit.
Let us know today what you think.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
the McFly Effect – n. the phenomenon of observing your parents interact with people they grew up with, which reboots their personalities into youth mode, offering you a glimpse of the dreamers and rascals they used to be, before you came into the picture.
I think this is interesting, and my kids likely experience this a bit since they’ve seen me with some people I grew up with. However, I don’t think I’ve ever experienced the McFly Effect because I have rarely seen my parents with anyone they grew up with.
I visited my Mother’s parents and family as a kid, but I barely remember that. Since then, no contact as she is estranged from them.
My Dad was born in another country and I’ve only seen him with family, of which he is the oldest, so I don’t get any view into him as a younger person. He always seemed too responsible.
I wonder how many others find this to be an interesting feeling.
From the Dictionary of Obscure Sorrows
Recently I was trying to use a connection string to connect in SSMS. There are some tools that have a connection string available as an output, including some Redgate tools. ADS lets me paste in a connection string. Can I do this in Management Studio (SSMS).
Yes, but be careful.
Getting a Connection StringThere are lots of ways to get a connection string. You can build one, or use a site like https://www.connectionstrings.com/sql-server/ I tend to get them for applications, as I work more with app developers.
In any case, I’ll use this as my string:
Server=Aristotle;Database=Sandbox;Trusted\_Connection=True;
If I open SSMs, I get a connection dialog like this one. I see the server, but if I wanted a specific database, I’d have to go to the second tab.
I could also go to the last tab, the Additional Connection Properties, and paste my string in there.
I press Connect and that works great.
If I open a new Query Window, I’m connected to the Sandbox database.
The ProblemLet’s change the connection. I’ll press the icon to the left of the database name in the image above. Then I’ll go to the second tab and pick a database. In this case, I’ve selected the Westwind database.
If I click Connect, I see this:
The connection string in the individual parameters overrides the selection here. If this were a day later, I might remember I’d put a string in the Additional Connection Parameters. I rarely use this and when this happened, I couldn’t figure out why this wasn’t working.
I thought this also happened with changing the main dialog and the server name, but this appears fixed. At least on SSMS 20.1, the additional connection parameters are linked with any saved systems I have on the first tab, so if I change servers in the drop down, the additional connection parameters link to the last entry for that server.
On my laptop, which has 20.2, I’ll enter this as a connection string in the last tab:
In this case, I have multiple container instances running on different ports. This instance is on port 41433. On the main tab of the connection dialog, I see this:
What happens when I press Connect? I get to this server:
The default port is SQL Server 2022, but the additional connection properties overrode the front screen in this case.
SummaryI’ve never had this problem in 30+ years of SQL Server work until this summer. However, it’s the first time I’ve really been focused on using connection strings more often than just entering values in the dialog. I only noticed this as I had a deployment going to one instance, but SSMS kept connecting to the other one and I didn’t realize this.
I think it’s OK to have conflicting values in locations, but it wasn’t clear to me that these values override others. This is documented on the MSLearn site, but it’s easy to miss this.
Hence this blog.
On a recent weekend, I got a text from my bank that they had declined a charge to one of my business accounts. I called them back and they let me know there had been a couple of weird charges on the account that their AI system detected. This seems to happen every year or two so I wasn’t overly worried. I cancelled the card and ordered a new one.
A day later, my wife got a call about our credit card with the same issue. She cancelled the card and got new ones ordered. However, I use that card to travel and I had a trip booked. Suddenly I was without a credit. Luckily, we have another card for my wife’s business that I could use. I called the bank and had a card expedited, but the situation created some stress. In fact, I panic-bought an RFID-shielded wallet. I’ve resisted for years, using an older, large wallet me daughter bought for me one Father’s Day that always reminds me of her. The timing across a few cards was weird, and I suspect my wallet got scanned somewhere and both card numbers were stolen.
A few things. First, be careful with the new tap cards, as they can be scanned and read from a distance, albeit a short one. Second, having a spare payment method might be nice in this age of non-cash transactions. Third, why is technology failing with new cards?
I lost a card last year and knew it was gone. There were no charges, but I couldn’t find it and needed a replacement, so I cancelled it and ordered a new one. In minutes the digital cards on my phone (and watch) had been replaced. I had new numbers and could transact business.
Why would this be different? The banks arguably have better knowledge of my digital wallets, and replacing those is much easier than relying on a snail mail server and the time it takes to deliver cards. In my rural area, we regularly have reports of stolen mail, with thieves targeting credit cards and physical checks sent by snail mail.
This was a minor issue in my life, and I am fortunate I have other ways to manage payments in this minor crisis. I was (and am) happy that AI systems are often detecting fraud. I haven’t had any fake charges go through in a decade and almost every real charge is approved, even with my crazy travel schedule. However, I’m disappointed in technology in this case.
Many organizations are engaged in a digital transformation. They’re hiring software developers and trying to take advantage of all the data they have to improve services and efficiency. Security, service, and spending would be served better with a little technology improvement here.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I have often made an effort to attend conferences in the past to grow my career. Even today, when I speak at a conference, I’ll try to go to a few sessions and learn something, but I can be distracted. It’s rare I focus on just learning stuff without other responsibilities.
I’m at Small Data today and tomorrow doing precisely that. I’m just an attendee, and today is a couple of workshops for me with a variety of talks tomorrow. It’s a real career growth opportunity for me and I’m excited. I’m doing the Data Warehousing and Design workshops today and sitting in sessions all day tomorrow.
I can’t remember where I heard of this conference, but when I saw the manifesto and sessions, I was intrigued. There were tentative plans this week for me to do a customer visit, but when that got delayed, I jumped on the opportunity to visit San Fran and learn something.
As I’ve worked with a lot of customers, I see the value of small data sets providing lots of agility for teams, while also allowing them to get work done, as long as the data sets are relevant and representative. That’s a big part of me pushing the Subsetter at Redgate.
In this case, the conference focuses more on analytics and AI, and likely more developers than DBAs, but I think it’s a chance to get different perspectives, maybe learn a few things, and perhaps get others to see my data viewpoint in the hallways discussions.
Every once in awhile I hear about someone in law enforcement sure that tech people can build in a safe, secure way for data to be unencrypted by the company or vendor. The latest appears to be from Australia, where the Security Intelligence Organization wants tech companies to build this into products.
Backdoors never work. Anytime an encryption key is stored, it could be stolen. We see this all the time. Keys are just data, and companies lose data all the time. At scale. Governments are certainly not immune from this. One of the reasons that Azure allows a BYOK (bring your own key) for encryption mechanisms is that many organizations don’t want to trust Microsoft to store their keys. I’m guessing Microsoft doesn’t want the liability, either.
Many of the data protections an organization might implement are outside the scope of data professionals, but we certainly have responsibilities in this area. We should certainly manage our keys appropriately, and my recommendation for many people is that they have a separate repo and pipeline for the deployment of privileged objects, such as encryption keys. Your organization also ought to have a process (and test it) for key rotation. Keys expire relatively infrequently and the person who last rotated the key might not be available when you realize this process needs to be completed.
It’s especially important everyone knows how this works for those emergency situations where something expires and none of you realize there’s a problem until a client calls.
There are plenty of other security mechanisms we ought to be using. Secure your backup files and limit access to shares or even processes that move these files around. In general, limit access to those who need access to everything. Using organizational groups is the best way to do this, along with regular audits to ensure that those who change jobs are removed from groups that aren’t needed. I haven’t seen any organization that has a good process for knowing what positions need what access. I often only see people given new access for a role change, giving them the new roles that match some other employee. This is usually done without any previous access removed when it is no longer needed. The most senior people often have the most access, not because they need it, but because they keep getting new roles.
Managing security with roles too granularly can become a nightmare, though ensure you use roles everywhere you can while trying to limit the numbers of roles. For most databases, we are giving access to nothing or everything, but there are reasons to limit access for certain data. You might consider two roles by default in most places: one for privileged users and another for everyone. Easy to ensure new objects get grants to one or both roles and let access be managed by role membership.
Keep it simple, but keep it secure.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
mottleheaded – adj. feeling uneasy when socializing with odd combinations of friend and family, or friends and colleagues, or colleagues and family – mixing a medley of ingredients that don’t typically go together, which risks either watering down your identity into gray much or accidentally triggering some sort of explosion.
Mixing groups of people from different situations has sometimes been a struggle for me. I’ve felt quite mottleheaded at times when I have people I work with in coaching, at Redgate, and from friends/neighbors/family. It seems that they can be very different people.
I’m sure I’m overthinking this, but I’ve rarely invited different groups of people to the same place, precisely because I do worry that someone might not enjoy the time with those others. I know I’ve felt awkward at times at horse-related things for my wife. I don’t mind and am happy to support her, even if it’s strange for me.
From the Dictionary of Obscure Sorrows
I don’t see a lot of SQL at The Daily WTF, but this one was great. It’s a stored procedure that was likely just converted from embedded code, as noted by the poster. It’s a strange set of code, that doesn’t quite make sense to me, and I can’t imagine why someone wrote it. Arguably, this is no better than having this code in a C# or ASP.NET application.
Or is it?
I think it is better. If I saw this code in a review or even in a production database, I could work on cleaning it up, adding protection against SQL Injection, and even tuning how it works to reduce the load on the database. I could likely wrap testing around this and get it deployed way quicker than if I were trying to update the source code for an app. More importantly, this is centralized code. If this is called from multiple places in the app code, I’ve fixed it once, not requiring an app developer, who has other work being piled on them, to spend time updating repeated sections of the code.
Even better, I could refactor some of the schema behind this stored procedure and easily find that my changes might affect this code. I could add a feature flag to this procedure and slowly migrate my schema in the background, without disturbing the user, again because the code is centralized. That’s a technique that most developers use in C#/Java/Python/etc., so why not in SQL?
I find it very interesting that a lot of developers refactor their classes and methods to better adhere to SOLID or some other practice, and they are happy to remove repeated code in their language. Yet, they don’t want to implement a stored procedure or function into their calls, essentially creating a database method for the things they need.
The more I work with legacy systems, the more value I see in using stored procedures. Every developer ought to know how to build them, and every developer ought to be able to create them in dev systems so they can easily deploy database and code changes together. More importantly, they can also share the load of tuning queries with operations staff, who may notice things in a production environment that are not apparent in development ones.
The big challenge in all of this is that database tooling is immature. Capturing your database code in source control is hard, and often it is a separate process from the one you follow for application code. I see some companies (including my employer) trying to make this easier, but there is a long way to go, and a lot of habits to change for developers.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
“We need to get the code written for feature X. Can you finish this query today?”
We’ve all heard some variation of that request. We have a request or demand, and we need to get it done. We need to get code out so our business can advance, sell more things, get more customers, etc. There’s always some reason to get new code pushed to production quickly.
However, many technical people want to ensure their code works well. At least, I believe most do. While most people can write code that works and meets the requirement, some don’t know how to write code that performs well or don’t know how to test their code to check. Often there isn’t a large workload in dev or test environments to verify things.
There may not be a large workload in production either, at least not at first.
So, what do you worry about first: your code being used or performing well? That’s a similar question to this one: Worry about Scalability or Popularity First? While most of us don’t work for a startup and our organizations have some sort of financial stability, does popularity matter?
I’d say that for any feature you build, whether a startup mobile app or a legacy ERP system, you’re still looking at this type of question. You want to know if it’s used, and how often. That might determine if you spend more time on this feature or area. Maybe you have some idea of popularity, or just plan old use of the feature. In that case, certainly make sure it will scale to not only meet your data size now, but plan for some level of growth across the next 6-12 months.
If it’s a new area of functionality for your application, then maybe you have no idea. In that case, the DevOps approach is get something working, a minimally viable version of your code or query, and then tune it later if it becomes a problem. Many technical people approach the endless number of tickets and requests they get like this.
The problem is management often doesn’t budget in time to clean up the technical debt (to care about scalability).
My view for database code is that we should always be leveling up our database code knowledge. If we deploy bad code in production, and can’t fix it, then at least we can avoid adding to the problem by writing the same poorly performing code again. Learn a better way to write that type of query. Whether you’re splitting strings, finding islands and gaps, calculating running totals, or anything else. Learn what works well and write that code next time.
That helps your team balance the scalability and popularity-chase by producing good code the first time. Or at least, the next time.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I saw a post internally that asked this question: Anyone have a handy powershell script testing if the installed flyway version matches a specific string?
That seemed simple, but getting program output from PoSh wasn’t something I’ve tried. So I tackled the challenge and this is what happened.
Getting the OutputThe first thing I wanted to do was actually figure out what the output of checking the version was from the CLI. I looked at the help and noticed a version verb. When I run that, I see a bunch of lines of output.
A lot of output. I need to parse a bunch of strings, and then find a line.
My first experiment was to run this to get a file with this output.
flyway version > fwversion.txt
Now, let’s parse this.
Parsing ContentIt’s been awhile since I read stuff from a file, but I know Get-Content works to read the file. What about finding a line. I saw this post with an answer that noted Select-String can be used, so I decided to try that.
Here’s a first cut of code:
That didn’t work. However, with some experiments, I tried this code:
Get-content fwversion.txt | select-string 'Edition'
That worked.
Now, I’ll assign that to a variable with this code:
$a=Get-content fwversion.txt | select-string ‘Edition’
Next, I’ll split this string by spaces into a new variable with this:
$b = $a -split(‘ ‘)
Then I can evaluate the various element of $b. You can see below the first and third elements are what I’m interested in. Really the third. Remember, PoSh is zero-based.
That let’s me parse the output, but I don’t want to save a file. Now on to the next step.
Capturing the Output from a ProgramOne of the things I know you can do in a PoSh ptompt is run a program. The redirection operator allows you to move output. When I tried it, I couldn’t quite get the output I wanted, but I did find this post that helped. With that, I ran this code:
$a = & "flyway" --version 2>&1 | select-string 'Edition'
This runs Flyway, captures the output in a stream and then uses the code above to find the right line. I assign this to a variable.
Almost there.
Adding a Parameter and a TestSince I want to call this from the CLI and pass i a parameter, I added a param() clause to my script and then a test that compares the version output from the flyway.exe to the parameter. That gest me this code:
param( [string]$versionToCheck="")$a = & "flyway" --version 2>&1 | select-string 'Edition'$b = $a -split(' ')if ($b[3] -eq $versionToCheck){ Write-Output("$($b) installed")}else {
Write-Output(“wrong version – $($b) installed”)
}
Now I can call this from the CLI and check things. It works well. At least for now.
I am certainly not a PoSh expert, but this short script took me about 15 minutes to write with a little research. Then a little testing and I sent it off to the requester. Haven’t heard any complaints, so I’m hoping this actually works for them.
There can be a big divide among tech professionals on how they view data privacy. Some don’t worry too much, and some are very upset about the lack of data privacy and poor data handling practices from many organizations. Most people are probably in the middLe. I do know that every time I post about a Tesla, there are many people who mention the incredible amount of data Tesla collects, and how that makes them nervous.
Tesla discloses that they collect, and I get copies of my data with my own logger. I find the data interesting and I glance at it over time to look for trends. I’m still happy that my top destination in the last year (after home) is the gym
Other car companies collect data on you and GM is in trouble in Texas. I suspect other companies do this, likely to learn more about how to build better cars, but also to get data they may be able to sell. The same thing likely happens with most of your data. Certainly, mobile phone companies collect lots of data, and I wouldn’t be surprised if a lot of your rentals/purchases/visits are being captured and sold to others from all sorts of vendors. Some of you might even be capturing usage data in your software that your customers might not like.
I don’t know how I feel about this. I do think disclosure is important but not in the form of some EULA or a contract that people have to click through. I think there ought to be choice of what data is being collected and how it’s used. Or if it can be used. While I often just accept or dismiss cookie banners, I do appreciate that I can refuse cookies and still use more websites. I ought to be able to easily opt out from most data collection from most companies. It really should be optional.
The world isn’t going back to analog devices and actions. Digital technology, with software invading most parts of your life, is here. I wouldn’t be surprised to find out most new appliances have a phone home feature that captures data and sends it back through unsecured wi-fi. A good reason to put a password on all your networks.
In portions of the world, there are restrictions on data and privacy. They could be stronger, and the laws could limit more of what companies can do. Many companies still make money or run fine.
They will be fine. Organizations always find ways to work within whatever system exists.
I think that would be a good thing for most of us if more data privacy restrictions were in place. However, I don’t know how most of us would even go about helping enact them.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
How can I quickly get a CU patch for a system that’s out of date? I’ll discuss that situation.
You might think you get to patch every instance every few months, and you may be able to. But most of us have laggards in any decent-sized estate. Someone always wants to avoid patching, or skip patching on the day you’ve scheduled every other system.
This is part of a series of posts on Redgate Monitor. Click to see the other posts
Tracking VersionsThe Estate page in Redgate Monitor contains quite a few different views of your entire estate. This section is designed to aggregate data across all the systems you are monitoring. If I look at https://monitor.red-gate.com/Estate/Versions, I see this as a default.
This gives me an overview of what SQL Server versions I’m monitoring. As you can see, our test estate has a mix of versions, from 2008R2 through 2022 and one Managed Instance. The counts are in the pie chart, and we can see how many are up to date in the bars to the right of each version. As you can see, lots of our estate needs patching.
I can also see at a glance for each version that the latest update is and its release date. The download link is a quick way to download the latest patch from Microsoft. We maintain this list and Redgate Monitor will update it on a regular basis.
When I scroll down, I see the details of individual instances. These are grouped, though I can change that with the toggle in the upper left. For each instance, I have the name and the major version,
The current status and the latest patch are listed next to each other, with an icon and color coding that let’s me know I need to upgrade (yellow up arrow) or I’m patched (green checkmark). I once again have the latest patch, linked to the MS article as well as text that let’s me know how out of date I am. You can see these patches are a month old. Auditors are potentially OK with that.
However, for my 2012 instances, I’m way out of date. No excuse for that. We keep these firewalled and protected, and they are available here only for demonstration purposes.
To the right of this we have our support dates. If a date is past, we mark it with a triangle to let you know that you have unsupported versions. That may or may not be an issue for your organization.
SummaryThis section of the Estate tab isn’t something I expect DBAs or sysadmins to check often, but I would schedule a reminder to do this quarterly. Knowing the state of our patching process is important, especially when there are security updates being released. We have had a few in the last year for SQL Server, and it is important to patch and apply those.
Seeing not only the status, but having an easy download link makes this a very handy tab that I wish I’d have had in quite a few jobs during the 90s and early 2000s. If you haven’t checked your estate tab in Redgate Monitor, you might do that today.
Redgate Monitor is a world class monitoring solution for your database estate. Download a trial today and see how it can help you manage your estate more efficiently.
The fifth episode of Simple Talks is out.
Simple Talks is the Redgate podcast from myself, Grant, Ryan, and Louis. The main page is here, and it has links to the audio versions as well as a the video one.
This episode was recorded in Austin, where we celebrated the Redgate 25th Birthday as a company. Grant, Ryan, and I grabbed a conference room. Ryan and I also grabbed some cowboy hats that were in the office.
This was a fun one, as both Ryan and Grant have a lot more experience with PostgreSQL, so I was picking their brains on which is better in different ways.
We recorded a few more in Austin, which are being edited and scheduled. Look for those to come out in the next few months.
I recently installed SnagIt and it was annoyingly saving both PNG and SNAGX files. This post shows how to get rid of the SNAGX file. If you just need the solution, scroll to the bottom.
I installed Snagit recently with a new machine. The old screenshot software I had been using was banned by IT as it had an unpatched vulnerability.
The main reason I want a screen capture tool is for sharing an image. This might be for a note for me, feedback to a product team, a part of a screen for a blog or article, or even showcasing something for a customer.
What I have done for a couple of decades:
- hit a hotkey
- select a region of the screen
- move on
I expect to have the region of the screen saved in a folder on my workstation, where I can then grab it later and send it somewhere.
The SnagIt ProblemWhen I added SnagIt, I set up a automatic capture with these settings.
- select: region
- Effects: non
- Share: File, automatic filename, specific folder
- No preview, no delay, capture the cursor
- Preset – One set up
- hotkey: CTRL+Shift+B
- Image type set to PNG
This worked well, with my images being captured.
Except, with every PNG, I also had a SNAGX file. That was really annoying because if I edit, I’m editing the PNG and resaving it. I don’t need 2 image files.
Customer SupportI had a chat with customer support, who was not helpful. They suggested this article: Save to another format. I eventually got the person to understand I didn’t need that, and they said this wasn’t supported.
I complained a bit on Twitter, mostly just annoyed. The person who responded gave me the same article, but then they responded later with the solution.
The SolutionThe place to configure this isn’t the SnagIt app, which is where I’d expect it. It’s in the Snagit Editor. I’m guessing that the capture process calls the editor somehow, which is why this design exists.
Go to the Edit menu, then select Editor Preferences. Go to the Library tab. In there, uncheck the “automatically save new image captures to library”. It’s checked below, but remove that and it should stop with the SNAGX versions.
SummaryNo screenshots because, well, when I try to capture one the SnagIt app and Editor disappear. The one above is from Twitter.
This is a weird architectural design. The capture widget would be the place I’d expect this, but the “save new image captures” implies to me all image captures, not just the SNAGX format. It’s poorly worded, IMHO, for users to understand.
However, my problem is solved and I don’t need the job I created that runs every day and deletes *.SNAGX from my folder.:
It’s Labor Day in the US, the traditional end of summer for me as a kid growing up in Virginia. This was the last day of summer before school started for me. It was also the day when many of the seasonal businesses closed in Virginia Beach.
It’s not quite the end of summer for me, but it is a day of Labor. Travel and a strange weather pattern this summer have me behind on ranch chores, plus, the ranch manager AKA my daughter, is on vacation. Today I’ll handle the horse chores and then start fixing, building, and more today. There is always plenty to do, and as my daughter has taken over much of the day-to-day ranch management, she continuously sends me a list of things that need repairing. She’s happy to help, but I have to go over how to accomplish some tasks that she’s never done.
Hopefully it’s a quiet day for those of you working in technology, with no outages or security incidents and you can start the week slowly.
For those in the US, hopefully you have a great day off and enjoy a break from your labors.
Steve Jones
I had someone ask me about DuckDB recently. Would I think that’s a good choice for a database? I don’t really know. From their blog and some online research, maybe, but it’s also a minority player in a niche space.
I had a chat recently with someone that had implemented ArangoDB, a graph database. Why that and not Neo4J I asked them? Someone at the company had tried the database and recommended it. Not a bad reason, as I think experience with tech is important, but it’s not the most important thing.
As I’ve aged, and maybe matured, I think less about the ability of a technology to work and more about the ability of a technology to be maintained over time. Not by me, but by everyone in my organization. Not everyone, but can anyone working on our staff learn and use it, including the future employees we haven’t yet hired.
There seem to be no shortage of new niche technologies. I have a few newsletters I subscribe to, and I see new projects and new solutions appearing every day. New tools, utilities, frameworks, even databases. Some of these might be amazing, and incredibly useful, but will they exist in a few years? In fact, that’s a question I ask myself about plenty of Microsoft technologies that appear. Will they really be around in 5 years? Long-term, or at least medium-term, supportability is important.
I also worry about the training and learning required for new technology. I’ve seen companies that adopt too many products in their tech stack and it becomes hard to hire experienced people. Even if we hire smart people that can learn, we have a lot to teach them. The more we need to teach, the slower they are to be productive. It can be even slower for us to trust them to work independently, especially in a crisis.
I think that most organizations should limit the number of technologies they use. This could be frameworks, languages, and more, including databases. Don’t add something new just because a developer, DBA, or even executive likes it. Certainly, be careful about changing technologies when the change isn’t adding value to your organization. Every change has costs, every new advantage contains a disadvantage, and every additional thing creates training requirements. Some people might pick things up quickly, easily, and during their off hours. That person might be you, but how many others will be able to do that?
Not many. That’s been my experience. The world is full of average people, by definition. While the average level (skill, capability experience, etc.) at your organization might be higher than our industry, over time, that will change. As our organizations grow, and as we change staff, we often become more average.
Our choices, and methodologies, our architecture, and more must survive the average employee, not the high performing ones.
Every organization ought to limit tech choices. There ought to be a process and way to add new technologies, and employees ought to be able to submit a request, make a case, and have others decide if taking on a new technology makes sense. If so, great, but do so carefully.
I like seeing new technologies built and adopted, but I also try not to just adopt the latest shiny things. Experiment, in a time-boxed fashion, and make decisions when appropriate, consciously because the benefits outweigh the costs. And not just slightly outweigh the costs, but substantially. In all likelihood whoever proposes the new tech isn’t thinking about the downside, and there will always be more downsides than you can see right now.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
flashover – n. the moment a conversation becomes alive and real, when a spark of rust shorts out the delicate circuits you keep insulated under layers or irony, momentarily grounding the static emotional charge you’ve built up through decades of friction.
I sometimes wonder how sad or disappointed (or angry) the authors or this dictionary are at the world.
I don’t get too insulated with irony, but I do have flashover moments when I suddenly become interested in a conversation. It can be with a customer, when they have a cool problem, or they see some excitement in how they might adopt DevOps.
It can be with friends or family, when I hear them tell me something wonderful about their life.
It’s a cool feeling when you might be half paying attention to a conversation, or you feel like you’re having small talk, and all of a sudden you get excited or interested, more invested, in the chat.
I love that.
From the Dictionary of Obscure Sorrows
Redgate added Git integration to the free, Community edition of Flyway Desktop. I saw the announcement and decided to make this post to show how this can work for a new project.
We do need git installed, so head over to the free git download if you don’t have it. From there, install Git and you’re ready to go.
I’ve been working with Flyway Desktop for work more and more as we transition from older SSMS plugins to the standalone tool. This series looks at some tips I’ve gotten along the way.
Source Controlling Your ProjectWhen you start Flyway Desktop Community, you should see the edition in the upper left, as shown here.
I’ll click Open project, and choose one of my existing projects. When I do that, I see all the migrations in the project. I can also select or add a target and run flyway commands from here.
What’s new is the right hand sidebar, which now has the VCS controls. If I click the left arrow in the upper right, I get the sidebar to expand. I can see I don’t have any changes. This bar wasn’t available previously, but now it is.
Let’s make a change. I’ll close this (click the arrow at the top) and return to the migrations screen. I’ll click the “add migration” button (the arrow points to this in the image below).
When the editor opens, I’ll add some code. I’ll also change the name. Notice there are no changes in the right sidebar.
When I save this, all of a sudden, there is a single change in the middle of the bar.
Expanding the sidebar and clicking on the middle icon, I see my one change has been added as a migration script.
I can add a comment and commit this or continue working. When I’m done committing, I can easily push my changes from here to the remote.
I’m manually managing scripts in Community Edition, but I can do it all from Flyway Desktop, including all the version control work.
Flyway EnterpriseIf you want to get more from Flyway, try Flyway Enterprise out today. If you haven’t worked with Flyway Desktop, download it today.
If you use the CLI Flyway Community, download Flyway Desktop and get a GUI for your migration scripts as well as version control.
Video WalkthroughNo video walkthrough this week as I’m on the road.
You can check out all the Flyway videos I’ve recorded.
I was working on some branching and merging with a customer and they wanted to move a file from one branch to another without taking the entire commit. I had to dig in a bit and see how to cherry pick a file, and not a commit. This post looks at how this can work.
I’ll do this in the git CLI. I’m sure it works in many clients, but when I do something strange or new, I like looking at the CLI. Mostly because when I make a mistake, the clients send me to the CLI often anyway, so I get comfortable there.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. You can see all posts on Git as well.
A Simple SetupFirst, grab a repo. In this case, I had a branching/merging repo I use with customers to show some things. I’ll use the repo at: https://github.com/way0utwest/BranchMerge
For me, I have a main branch, as well as a dev and qa branches. There are likely some feature ones as well.
To start with, let’s check main. Everything is up to date (I pulled already):
Let’s also check QA. Same thing.
I’ll now make some changes on QA, adding a file and changing two others. Once I do this, here is my status, pre-commit.
If I don’t commit, these files don’t exist, and I could change branches and add them there. However, once I commit, I might want to move them in a certain way. I’ll commit and my status is clean.
Now, I could switch to main and do this to move one file:
git checkout qa -- README.md
However, I’m just doing that without review. So, I wouldn’t do that and you shouldn’t either. Instead, let’s create a PR.
Using a Branch for Peer ReviewA PR is a pull request, but it also means we ask for some peer review. In my case, I’ll use this code to create a new branch and then pull in two changes: my modified readme and one of the other changes.
``` git checkout -b r9-qa
git checkout qa -- README.md
git checkout qa -- "V6__create proc gettwo.sql" ``` Ignore my typos, but I’ve run 4 commands: checkout 3 times and status once.
Now I’ll commit and push these changes to my r9-qa branch.
Once I do that, Github detects a push and asks me to create PR. I do, and I see the PR with these changes.
Now I can proceed with my flow, and my cherry picked changes are captured.
There is a git cherrypick command, but often I find I need random files from multiple commits, while ignoring others in the commit. This works well for database releases.
A few references:
- git checkout
- How to cherry pick only changes for only one file, not the whole commit
- Git cherry-pick file from another branch
SQL New BloggerThis post took my about 20 minutes to write. However the learning and experimentation took over an hour as I read various links and dug into the docs and various posts.
This is a very useful skill, and one you can discuss in an interview. Write a similar post and you’ll be prepared for this type of question in an interview.
I’ve heard of Kafka before. I know it’s an Apache project and you can download or read more at https://kafka.apache.org/. I knew it was a way of moving data around, some sort of ETL tool useful for moving things around. More like a message and queueing system, which is a tool that seems like a great idea, but one that everyone struggles to work with.
And one that seemed complex. The overview is that Kafka is “a distributed system consisting of servers and clients that communicate via a high-performance TCP network protocol. It can be deployed on bare-metal hardware, virtual machines, and containers in on-premise as well as cloud environments.”
Would I need that or use it? In a lot of my database work, I’m not sure that it would easily fit into most of the OLTP applications or data warehouse systems. Maybe. Hard to tell. Their description of event streaming and the definition of an event make it seem this is a catch-all system for moving log data around. One that be so open-ended that it ends up requiring a lot of configuration for “my” system.
Here’s their definition of an event: An event records the fact that “something happened” in the world or in your business. It is also called record or message in the documentation. When you read or write data to Kafka, you do this in the form of events. Conceptually, an event has a key, value, timestamp, and optional metadata headers.
Recently I watched a Kafka presentation at THAT Conference (which was a fantastic event). In the talk, this sentence caught my eye: “[Kafka is] a pipe to move data from A to B, C, D”. I’ve certainly had that need, and sometimes configuring lots of pipes is work. If you’ve ever worked with replication and the publisher/subscriber model you likely get a twitch in your eye if a ticket is opened to configure a new subscriber. Not because the configuration is hard, but because the ongoing admin can be a pain.
The talk dives into some of the complexity of designing and implementing a Kafka system. For developers that might write to the stream or read from it, things seem simple. For admins and architects, less so, and I can’t help what happens when a reader goes down. I have nightmares of replication subscribers being down and transaction logs not being reused.
Kafka doesn’t seem as complex as I thought before, but it certainly doesn’t seem simple or easy. Kafka is not a panacea for moving data around, but it is a well-understood and widely used technology. Those things mean more to me now that I find myself considering the challenges of maintaining a system over time and hiring staff who understand it. It’s something I’d consider using in the future, and maybe something I’d like to experiment with a bit more and learn how it works at a more practical level.
If you use it, or know more, I’d be interested in how well Kafka has worked for you, either as a developer or admin.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
As I’ve been working with some AI (Artificial Intelligence) technologies, what I’ve often found is that they produce junior-level code. The code I’d expect from someone early in their career or inexperienced in a particular area. That is code that likely works, but isn’t efficient or clean or perhaps incomplete in some way.
I’m sure AI technologies will improve, and we’ll be able to train them better for our environment. Just like we train junior developers to be better. However, what does that mean for junior people across the next decade? I ran across an interesting post on the death of the junior developer, which speculates we might have a problem as an industry.
The post references an article from Gene Kim, where a law firm sees a similar problem with their junior people, who are associates. That position might be equivalent to the junior developer in software. Someone with more experience and knowledge often reviews work and helps shape it, even though the junior person does the work. With AI, however, we might not need the junior person. Instead, the AI produces work the senior person has to review. Finding issues with associate work is a lot like finding hallucination problems in AI responses.
The same could be said of coding. There is plenty of poorly written code, but if senior people become good at writing prompts and getting the same code back that a junior developer would write, then how many junior people do we need? Arguably less, though you might still need a few. Or you might think that you need all the junior people and you’ll get 10x more work done, clearing your backlog. Certainly, I know most developers, DBAs, and other IT people have a large backlog of problems.
However, the problem with junior people using LLM (large language model) AIs and getting more done is that they might generate a lot more bad code, so much that your senior people can’t find the time to review the code and you end up with systems that contain even more technical debt than you have today. Perhaps we even find systems that don’t perform well enough for regular use or create constant issues that your developers try to fix with AI, which might not work. I can certainly see things deteriorating rapidly.
There’s a great quote in the Gene Kim piece: “I believe this furthers the case that AI helps the experienced people far more than inexperienced people — the seniors more than the juniors.”
I’m starting to think that might be the case. Senior people are going to become very productive, and very valuable. Junior people are going to struggle, and while they’ll get work done, the quality will vary. Maybe that’s good, or maybe we will start to see a rapid divergence of not only productivity but salaries. If you can hire a senior person to produce better code at the same rate as your 5 junior people, maybe you’ll want to pay that senior person $200k a year and reduce junior rates to $45k a year.
I don’t know that we’ll see rapid changes, as many organizations are slow to alter the way they hire, code, or structure their staff. However, as there is success by others, especially when it’s touted in places like the ETLS, I can see other managers being influenced. That will filter over time to those who hire to pick the productive, senior-level people who can showcase some code skills in an interview. Craft a prompt to solve a problem, get some code back, refine it, explain where and why you’d change your prompt or use parts of the code, and we might see the AI-capable people getting hired quickly, and for fantastic compensation.
I don’t know that I think this reduces a lot of junior staff, mostly because of organizational inertia, but I do think that learning to be better at your craft and learning to use AI is likely to increase your future earnings.
Steve Jones
As part of my job, I needed to research how a few things work with Synapse and Fabric. The latter includes the former, mostly. I decided to setup a workspace and a do some experimentation.
My first stop was the Stairway to Synapse Analytics. I started with Level 1, since I needed to create something.
In the portal, I created a new Synapse Analytics resource. I won’t go into details, but basically create a new resource and search. Read the article to see how this works. From there, I used a resource group I had for work stuff. I also had to create a new Data Lake Storage Gen2 account. I didn’t bother with the Managed Resource Group.
I added a user account and password and then clicked Create. I went to the summary page, where I could see the small cost for serverless. I clicked “Create” again to start the deployment.
The deployment failed.
I was leaning towards this not being helpful, but I went into the details of the error message and found something interesting. My subscription couldn’t deploy in this region (westeurope).
OK, go back, recreate the resource in UKSouth. That worked fine and I saw my resources.
At this point, I had a resource.
Adding DataThe next step for me was to add some data. I grabbed a few csv files and uploaded them to my storage account. Lots of querying in Synapse is through external tables to flat files, so I picked some files I can query.
Once these were uploaded, I was next interested to see if I could query this data. In my main Synapse resource, I see some endpoints.
I copied the serverless one and then opened SSMS. I put this, my user and pwd, and got connected. I was hoping that @@Version would tell me I was connected to Synapse, but I got the same results I get some Azure SQL Database, albeit with a different timestamp. However, ServerProperty() helps.
That works, what about my data? Let’s try a query from Level 2.
I’ll take this query and run it. I’ve adjusted this from the values in Level 2, which actually uses the Synapse Workspace explorer online.
–retrieve data from csv file
SELECT
TOP 100 *
FROM
OPENROWSET(
BULK 'https://synapsesqlprompt.blob.core.windows.net/sqlpromptfs/solar\_2024\_01.csv',
FORMAT = 'CSV',
HEADER\_ROW = TRUE,
PARSER\_VERSION = '2.0'
)
WITH (
[Time] DATE,
[System Production (Wh)] VARCHAR (100)
) AS [result]
It fails.
Hmmm, let’s try that in the browser. Here it works.
The error is with a credential.
There is a quickstart online, and that query works. However, there is a note in the query that I need a credential if my file is protected.
The article had this link and used the sample code to create a credential:
CREATE CREDENTIAL [https://synapsesqlprompt.blob.core.windows.net/sqlpromptfs]
WITH IDENTITY='SHARED ACCESS SIGNATURE'
, SECRET = '';
GO
Now I can run my original query and it works.
That was a pretty cool exercise for me to get started. In less than 30 minutes I was able to create a Synapse workspace, add some data, and query it.
Now to learn a bit more about how this works, and to use Flyway to deploy some objects.
It was just over a month ago that I got a Dell Latitude 7450 from our corporate IT group. It wasn’t my first choice, but as Redgate grows, they’re trying to be more secure and standard. I didn’t have a good reason to not try it, to I agreed to give it a try. My boss said if it was a problem, we’d get an exception and I’d choose another one. I had hoped for another HP Spectre, but this really is just a toaster to me. I need it to just work.
The DesignThis is a bit heaver than my old Spectre, but also larger. It also has more ports.
I got the laptop, so it doesn’t fold over into a book or tablet or tent. I had thought that might be useful at times, but in 8 years of two Spectre versions that opened past 10 degrees, I think I’ve done that 5 times.
The screen is nice and works well for me. I like that I have both USB C and USB B ports on the laptop. I wish I had one of each on each side, but I have 2 Cs on the left and 2 Bs on the right. It’s a very minor gripe.
I do like I have an HDMI port built in, which is something my Spectre was missing. This means I can take something else out of my travel kit.
I got this with 32GB of RAM, which is probably a good reason why Windows 11 runs smoothly here. I also got a 1TB drive, which is overkill. I had a 512 in the last machine and after 5 years, I still had space.
The keyboard is OK, fairly standard, backlit, and responds well. The trackpad is large, and seems to work well with the left/right clicks.
The touchscreen is good and it also works with a pen, and I got an Active one from Amazon that I’ve used in whiteboard sessions with customers.
AnnoyancesThe big thing was the trackpad for me. It is a little sensitive and if a finder or hand brushes it, I get some weird actions. Mostly I do something the causes windows to minimize, which is really annoying. I’ve tried tuning down some of the actions, but I need to do more. It doesn’t cause too many problems, but enough that it’s on my mind.
The Home and End keys moved to the top row. They were on the side of my old machine, and I was used to those. I still can’t quite hit home and end without looking, or without hitting insert. I also lost my page up/down keys as well. I can use Funtion+up arrow (or down), but that’s not something I think about.
ConcernsI’ve had 2 blue screens coming out of a closed machine in a month. The system was unresponsive once and when I tried to CTRL+ALT+DEL it blue screened after a few minutes.
I’m keeping an eye on it. This might be my machine or this model, but if I get a few more, I might need to replace it.
OverallOverall I learned that I don’t really care anymore about a laptop. It’s a tool, and I need it to work, but I’m not that wedded to the hardware. They keyboard matters to me, but of a bunch I tested, they were all find enough. Small differences, but overall they worked.
Other than that, this thing works well and runs fine. I still think some of the Win11 stuff that changed wasn’t an improvement and I need to figure out if I can turn some things off, but it works well.
I’d get another one of these without complaints. Assuming the blue screens stop.
I know a lot of people in this business do not have computer science degrees. While some do, I suspect it’s a minority. I’m certainly curious, so if you want to share your education experience in a comment, tell me if you have a degree and what the focus was, as well as answer a few other questions.
At DevOps Days in Minneapolis recently, professors Fox and Sen from Macalester College talked about their computer science curriculum for growing the next generation of professionals. Along the way, they also asked the audience these questions:
- what were you required to learn?
- what courses were key?
- what topics were the focus?
- what was the teaching style?
- what was missing?
While these are good questions for any curriculum, these are interesting points to reflect on for any sort of learning. If you learned about technology in the military, on the job, or by yourself, what did others (or you) think was important and required? What did you feel was left out of your learning?
Of my small group of 4 that chatted about this, only one of us had a CS degree. I started in CS, but I actually have an economics degree. I switched to business, sensing more opportunity in the 80s there. However, I’ve continued to learn, even taking some classes post-graduation, that helped me learn more about computing.
It was an interesting look at a modern CS major, with lots of comments from the audience. The professors left us with their questions about how to look to the future, address AI, and even if they should teach an operating systems course, something both feel is missing from Computer Science at their college.
I think computer science is important for the world to advance how we build systems, but a lot of the deep theory on topics isn’t something that many of us need to learn. I would like to see a software engineering major come about that emphasizes more of the knowledge we’ve learned about building software, with emphasis on version control techniques, software design architectures, and process flows. While DevOps has been amazing, a lot of that knowledge could be used to better teach people how to build software in different ways. Distributed systems, database theory, performance measurement, and even different ways of managing code are things that need deep treatment, not just a module in another course.
Let me know today what you think of your education in computers.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
A customer was having some trouble getting started with Azure DevOps (AzDO) and building their database, so we took a step back and decided to create a simple test pipeline, so they could get a feel for how things work and then move on to more complex builds.
This post looks at a basic pipeline on a Windows agent. I assume you have an Azure DevOps account and have created a project.
This post is part of a series on Azure DevOps. You can click the link to see other posts.
SetupWe decided to start with a very simple test to start, using a task to get a directory listing. That’s a nice simple task, and one you can re-use a lot as you try to configure your pipeline.
We created a new pipeline in our Azure DevOps project. To do this, we went to the Pipelines section of AzDO and selected “New”.
We next had a screen that showed choices. If you pick a repo, this wants to build a code-first, YAML pipeline. Great for coding and automation, bad for beginners. Select “Use the classic editor” at the bottom. Don’t worry, you can always move to YAML.
This takes us to a repository screen. We were working in Azure DevOps, so we can use Azure Repos, but if you used GitHub or some other Git repo on the Internet, you can choose that.
You can choose to build from other branches, but I tend to start with main, though we often protect main to prevent direct commits. Once you select the repo and branch, click Continue.
We now get to choose a template. For teaching someone new, I Start with an empty one. Click Empty job to move on.
This gets us a basic pipeline. To orient you, there are multiple tabs, Tasks, Variables, Triggers, Options, History. We don’t need any of these to get started. The Tasks item is highlighted, so we see our tasks.
The pipeline is the top level element, and most listed ot the left. The “Get Sources, is next, and is part of the pipeline as it downloads the repo. The Agent Job 1 is the container for a series of steps in our task, like a SQL Agent Job that contains steps. We’ll add different steps below here.
If you look to the right, you see some edit boxes. These are the top level pipeline settings. The Name is shown at the very top of the image, near the breadcrumb. If I edit this to say “Basic Directory” then this appears at the top.
The Agent pool is the list of possible agents. This defaults to using the Azure hosted agents from Microsoft, but you can create local agent pools if you want things to run inside your infrastructure on VMs instead. I’ll leave this, and I’ll also leave the Agent spec to Windows 2019. There are other options, which you can see.
Note, I am not a fan of using Latest agents. Pick a version that reduce debugging issues. Change this to a later version periodically when you are ready.
If you click on the Agent job 1, you see some different values on the right. You can rename this agent, or specify a different agent from the pipeline. We’ll leave this alone.
On the line with Agent job 1, there is a + to the right. This is how we add tasks (steps) under our agent. Click this.
This brings up a list of tasks to the right. There are a lot of ones listed, and you can add more to your account. We’ll live with this list.
We could scroll or click the various categories (build, utility, test, etc.), but let’s search. Type “cmd” in the search box. We see a limited list of tasks. We’ll click Add next to the Command Line Script.
Once this chosen, it is added to the left under Agent job 1. Click this line to see the various settings on the right.
Let’s configure this. First, we’ll change the name to directory listing. Next, let’s delete everything in the script box and type “dir”. We should see this.
Now click “Save and queue” at the top. We get to enter a save comment and can configure the run. Just leave the defaults.
Note: I grabbed this shot, and realized I’d accidentally set the agent to MacOS. I reset that before I continued.
Once we click Save and run, we should go to the screen that has meta data about this pipeline run. It looks like this. Click the line with the clue clock and Agent job 1.
That should bring up a log listing of the various steps. The listing on the left matches our pipeline steps. Whichever item is selected on the left filters the logs on the right to that info.
If we click the directory listing step, we see some info logged and then the command being run.
This directory listing matches what’s in our repo. We can see this easily by clicking on the repo.
When the agent runs, the software runs inside the folder of the repo that is downloaded as part of the pipeline. If you want to get another folder listing, useful sometimes as you configure things, just change the dir command.
SummaryThis is just a basic look at a pipeline and how to set up a simple task. It’s a useful task, and until you have your pipeline working, I’d leave this task in there. Being able to get visibility into the agent machine is often very helpful.
I’ll look at a few other options in future posts.
I had a customer that was concerned about the fragmentation alert for indexes and wanted to know how to change it. This post discusses the change.
This is part of a series of posts on Redgate Monitor. Click to see the other posts
Finding Alert ConfigurationThe settings for Redgate Monitor are available with the gear icon in the upper right side the menu area. You can see this below.
Click this to get a list of settings. The second section below has the alert settings, which you can see here. Click this.
This brings up a long list of alerts that are a part of Redgate Monitor. Scroll down a bit and you will see a fragmented indexes alert.
If you click this, you may see this, as the alert is disabled. This has not proven to be a very useful alert for most organizations, so it’s off. However, you can click the “customize” radio button.
When you do this, you should then see the alert settings. This is defaulting to all your servers, but you can customize the level of alerting by clicking on the left side to a server or group. I’ll cover that in another post.
The key things here are that you can exclude read-only databases, as the fragmentation won’t change. Below that, you see there is a medium alert specified at a default of 60%. You can change the low/med/hi with the drop down as well as the percentage in the spinner.
If you want more than one alert, flip the “Use multiple alert threshholds” button and you’ll see three choices. I might suggest you not do this as if you worry about fragmentation, it’s likely only a low or medium alert.
At the bottom, you can set a minimum index size. Smaller indexes aren’t a problem, so keep this relatively high. You might have to decide if 1000 pages is really high enough.
Below this, you can set your notifications for this. In general, use the defaults and don’t customize this for an alert. The changes get hidden and people forget. Use the same settings everywhere.
That’s about it. However, read these thoughts from Jeff Moden before you get too worried about index fragmentation.
Redgate Monitor is a world class monitoring solution for your database estate. Download a trial today and see how it can help you manage your estate more efficiently.
A few weeks ago, I was sitting in the audience, waiting for my turn to speak at DevOps Days in Minneapolis. Just before me, Xe Iaso delivered a funny and thought-provoking talk on building a social network on a whiteboard. It was very well done and had me feeling nervous about following that session.
The talk is a bit of a satirical look at an interview Xe had for a company that tried to get them to derive an architecture for a large distributed system. It was interesting to hear Xe note that often we have architecture diagrams of what we’d like to have, but never an explanation of how we implement a large system, especially one that has to grow as our workload grows.
This talk was a nice analogy of how often we get into situations where many of us can’t believe our system was structured this way. We often wish we could completely redesign things from scratch, and we’d do it better. Why didn’t the previous engineers think things through?
Watch the talk. It shows how a lot of software is built. We build what we think we need, but when we get overwhelmed, or often when we get unexpected pressures from others, we make decisions that seem to be the best ones in the moment. However, a few years later, with a few of these decisions behind us, we realize that each of those choices was too short-sighted. We have a mess of software the seems cobbled together rather than well-engineered.
I don’t believe we can engineer everything well from the beginning. I also don’t believe in early optimization, mostly because I think we are asked to build a lot of things that are never (or lightly) used. Who uses the paintbrush in Word or the FACTDOUBLE() function in Excel? However, I do believe that we ought to write code that performs well the first time, following patterns we (should) know well. We certainly should assume that whatever code we write will see at least 10X more data in production than in dev/test, so prepare for processing more data.
I’m sure many of you know of a codebase and system that doesn’t run well. I hope most of you have another application that does run well and is fun to work on. Hopefully, the goal of your organization is to turn out more software like the latter than the former.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Here are the resources from my talks today.
Best Practices for Seamless Database Deployments
- PPTX slides
Architecting Zero Downtime Deployments
- git repo: https://github.com/way0utwest/ZeroDowntime
- PPTX slides
Some good questions today, which I’ll write a few posts on:
- What are feature flags
- How do you handle adding a clustered index
- How do you communicate with downstream groups on something like a rename
Probably a few more, but it’s been a long day and I’m tired.
A few of you out there might be data scientists who profile data regularly. Probably a fair number of you do import/export work and learn to check data values, perhaps with counts, distincts, or other aggregates. I don’t know if the performance tuners out there look at the skew of data or the details of what is in a query that needs improvement. However, all of you are likely familiar with data and trying to query it for some type of meaning.
One of the largest data breaches occurred with National Public Data. Troy Hunt analyzed the breach as a part of his work with haveIbeenpwned. The piece is an interesting analysis of the data, trying to determine both it’s legitimacy as well as what is actually included in the breach. It’s a fascinating read and I encourage you to look at it not just from the data analysis side, but also to be aware of what data about you is being aggregated and sold by companies.
The read is interesting as it is a bit of a detective story, digging through data in a folder, which is something I’ve had to do. I’ve had people in previous jobs just dump a bunch of data on me and ask me to load it into a database. Or a table. Often without them knowing what type of data it is, what formats, do files relate to each other? Are there multiple tables worth of data in a file? All questions I’ve had to ask myself (and answer), and similar to what Troy did to analyze the breach.
Data is very important to many of us, in different ways, but I’m often amazed at how few people actually understand how to organize data and ensure others can track the metadata about their data (what their data represents). I’m guessing this is why every person that gets an extract of data to load into Excel formats it in different ways.
In many cases, people want the ability to query data, but they prefer to just focus on one table that contains a lot of information. They don’t want to know how to “join” data together. I think this might be the reason we see so many views in databases, and why we have views built on views. Each new client of the database needs their own view structure.
The world of data is a mess, even inside an organization. Once we start moving data between organizations, it’s truly a mess. We might bemoan all the inefficiencies and work we do to move, change, and re-load data as custom, human ETL machines, but there is one great thing about this tangled web. It provides for steady, secure jobs for many of us with no end of work in sight.
Steve Jones
Many years ago I was training for a triathlon. I had competed in the Sandman Triathlon the previous year in Virginia Beach and wanted to do it again. I had a young child, work was busy, and I was struggling to find time to swim, bike, and run every week. One night, I was at a work event with a customer who was also a triathlete. He was much more competitive and successful than I was at competing in triathlons, and he told me I should just get up earlier and find time to train or ensure I spent time after work on training before I went home to ensure I was meeting my goals.
That sounded fine. Want to be better at something, then spend time on it. Certainly, that’s what I often advocate for your career. Spend time on your career.
However, if I get up earlier, then that means I’m more tired at the end of the day. I’ll go to sleep, or more likely fall asleep, earlier and miss time with my wife. She won’t like that. If I try to ensure we get the same amount of time, I’d likely shortcut time with my kid. There’s no magic way to find more time. If I take time to do one thing, I’m taking time away from something else.
The same thing happens at work. Our Chief Marketing Officer noted this at our global meetup recently, saying that too many people are adding new tasks or projects and letting other work fall away. They don’t mean to let other work drop, and sometimes that’s a problem, but the reality is that we can only get a certain amount of work done as a group, and if we add new work, old work gets lost. The same thing applies to coding software. We might get more work from a developer in the short term, but that falls apart long term, and it can be bad for retention.
Time is one of the most valuable resources you have in life. I see this more and more as I age, and you must recognize that it’s a limited resource. For a short time, you might be able to get more time by sleeping less (or working more), but those things mean you are dropping other things in your life. That often isn’t good for our health, relationships, or happiness over time.
If you want to do something more, or new, then you should consciously decide what to drop. You have to make decisions and choose what is a priority and what is not. The things that are not a priority might get dropped (or their time reduced). That’s a big part of growing and maturing, as well as one of the worst parts. Making choices is hard.
Decisions you make are rarely permanent. They are often choices you make for a period of time. You’ll make some great choices and those might be long-term or permanent. You might make bad choices, which hopefully are short-term, and then decide to make a new choice. Whether this is at work or in your personal life, make the choices that drive you forward, towards your goals, but with an eye on keeping a balance across all parts of your life.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
povism – n. the frustration of being stuck inside your own head, unable to see your face or read your body language in context, only ever guessing how you might be coming across – which makes you think of yourself as a detached observer squinting out at a lushly painted landscape, though to everyone else you seem woven into the canvas.
That’s quite a definition. I know that many people aren’t aware of their body language or their expression. I know that watching myself in recordings has taught me to be aware of what I look like. However, before I did that, I didn’t realize how I reacted at times.
I see this often while coaching, and it can be hard to get players to realize their body language and expressions impact other players. Or the coaches.
I think I have some povism, and often would wish there were a monitor or mirror that helped me see what I look like. Easier to do on stage, but hard to do in life. Mostly, I can only reduce povism by asking my wife or kids to tell me when I am not projecting what I want.
From the Dictionary of Obscure Sorrows
It’s just a couple of days away from SQL Saturday Denver 2024. This Saturday, come join me at the Lowry Convention Center in Aurora for some great talks. We are lucky in the Denver area that we have quite a few great speakers and talented data professionals, so check out the schedule.
Then register and come.
Come learn about SQL Server, about PostgreSQL, Snowflake, Powershell, and more. I’m looking forward to sitting in a few sessions, and I have a few of my own. I’ll be delivering these talks:
- Best Practices for Seamless Database Deployments
- Architecting Zero Downtime Deployments
I hope to see you there.
I was trying to do a little testing of our subsetter and needed a larger database. So I decided to ask Prompt + to help me with a little AI assist. This post looks at what happened.
This is part of a series of experiments with AI systems.
Getting More DataI had a Northwind_FullRestore from my experiments with Alex Yate’s quick PoC tool. I selected that database and asked for a little code. I asked the AI to give me 1000 more rows of data in the Customers table. I got this code.
-- This script inserts 1000 rows data entry into the dbo.Customers tableWITH numbersAS ( SELECT TOP (1000) ROW\_NUMBER () OVER (ORDER BY (SELECT NULL)) AS num FROM sys.all\_objects)--INSERT INTO dbo.Customers-- (CustomerID, CompanyName, ContactName, ContactTitle, Address, City, Region, PostalCode, Country, Phone, Fax)SELECT 'CUST' + RIGHT('0000' + CAST (num AS nchar(4)), 5), 'Customer' + CAST (num AS nvarchar), 'Contact' + CAST (num AS nvarchar), 'Title' + CAST (num AS nvarchar), 'Address' + CAST (num AS nvarchar), 'City' + CAST (num AS nvarchar), 'Region' + CAST (num AS nvarchar), 'PostalCode' + CAST (num AS nvarchar), 'Country' + CAST (num AS nvarchar), 'Phone' + CAST (num AS nvarchar), 'Fax' + CAST (num AS nvarchar)FROM numbers;
Then I decided to go big. I changed my prompt, as you see below, and got more data. My new prompt: expand this to add 10GB of data to this table
I got this code:
-- This script adds approximately 10GB of data to the dbo.Customers table-- Firstly, we need to calculate the approximate size of each row. -- For simplicity, we'll assume each row is ~1KB in size after considering nvarchar data expansion.-- Therefore, to add 10GB of data, we need approximately 10,000,000 rows.WITH numbersAS ( SELECT TOP (10000) -- This will generate 10,000 rows in the CTE ROW\_NUMBER () OVER (ORDER BY (SELECT NULL)) AS num FROM sys.all\_objects a , sys.all\_objects b)INSERT INTO dbo.Customers (CustomerID, CompanyName, ContactName, ContactTitle, Address, City, Region, PostalCode, Country, Phone, Fax)SELECT 'CUST' + RIGHT('00000000' + CAST (numbers.num + seqnum AS nvarchar), 8) -- 8 character CustomerID to accommodate large number of rows, 'Customer' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar), 'Contact' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar), 'Title' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar), 'Address' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar), 'City' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar), 'Region' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar), 'PostalCode' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar), 'Country' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar), 'Phone' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar), 'Fax' + CAST (numbers.num AS nvarchar) + CAST (seqnum AS nvarchar)FROM numbers CROSS JOIN ( SELECT TOP (1000) ROW\_NUMBER () OVER (ORDER BY (SELECT NULL)) AS seqnum FROM sys.all\_objects) seq;
The ResultsThe results were a bit underwhelming. I ran it and got this:
The problem here is that the CustomerID is a 5 character field. The comment in the code notes that this is now an 8 character field, but that’s not great as I might not want to change the size of the field.
I then asked this: change this script to ensure the customerID is less than 5 characters and alphanumeric
I got the same script, but the first line of the select was:
'CUST' + RIGHT('00000000' + CAST (numbers.num + seqnum AS nvarchar), 8) -- 8 character CustomerID to accommodate large number of rows
Now when I run it, I get a problem with postal code, which now has the truncation error.
My ThoughtsIt’s easy to say this was a failed experiment. I didn’t get working code. But I got a bunch of code that was close to what I need, in a fraction of the time that it would take me to write this, even with SQL Prompt. Then add in the fact that I can edit this code to what I need, which works, and saves me times.
I think this has potential for shortcutting some work and getting me closer to what I need quickly, even if it’s not perfect. If I’d have asked a junior dev to help me with this, I might still have to edit their code. Just as I do with my AI assistant.
I would hope most of you reading this know what SQL Injection (SQi) is and how you can prevent it. Or at least what patterns cause problems. If not, here’s a short explanation that is worth reading. If you have more questions, ask in our forums.
SQL Injection has been, and continues to be, a problem in many systems. In fact, I chatted with Mike Walsh recently after he’d published this post on an attack for one of his clients. He has some notes that explain how your database server might be vulnerable, as well as a description of a recent attack example. He also notes that many of you are responsible for protecting data, which is separate from other security mechanisms. You need to be sure you are protecting your data, even in vendor applications.
I’ve seen similar issues in the past, both in homegrown and purchased applications, where text fields aren’t checked and SQL is built by concatenating user input with code. I’ve complained to vendors, though often a short repro helps them see the problem and I’ve found many companies will patch systems, albeit sometimes slowly.
There are application firewalls that can help, and certainly limiting access to those users who need access is always good, but that’s not helpful when the application is something that many clients use.
The best protection is education. If you don’t know what to do, or your developers don’t listen to you, perhaps engaging a consultant like Mike will help. I’m amazed at how often people listen to an outsider when they ignore the same advice from someone they work with. That might be especially true for managers who are more concerned with doing more new work rather than fixing something that’s not quite working well.
Security is becoming a bigger issue in many organizations. Not because we might get fined, but often because our customers might decide to choose another service if we can’t protect their data. There are other choices these days for most of the services we provide, and many organizations are finding customers increasingly fickle and quick to leave. This might not be the case in business-to-business work, but it does happen.
We often won’t be perfect in our security and even if we are, our systems will change and new vulnerabilities or attack vectors will appear. We can work on the problems we know and improve security over time. SQL Injection is fairly simple to prevent, but it takes some education, some practice, and some code review.
All things good database professionals should be doing.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
This month’s invitation is from Mala Mahadevan, who has hosted 5 times. This latest one is one that is near and dear to my heart as I use source control most days and I think it’s important for database code.
I’ll explain why below, but I’m glad to see this as a topic. We have a wide variety of technical topics being proposed, but not a lot about software engineering as a discipline, which is part of what version control is. Not building features, but better managing your process.
If you want to host a T-SQL Tuesday, ping me.
Capturing Database CodeThere are many ways to capture code, but I work for Redgate Software, so I use Flyway. Since I work there, I get a paid version, but I work with customers all the time and see a variety of things. If I didn’t have Redgate tools, I’d likely use something like SQL Compare to capture off database code, since that’s easy. Or SMO through SSMS.
As for how I work with code, I use Git to store the code. Git is ubiquitous and I rarely find customers without Git. Sometimes the data teams aren’t using it or don’t know how, but that’s why I’ve written some Git articles on getting started.
I also try to work in branches, with a protected main branch. This means no one can commit code to main, but rather need to commit it elsewhere and use a PR (pull request) to move the code into main. I do this with a lot of customers, helping them understand how to use version control to manage their code.
For my git work, I primarily work in GitHub in public repos. I’m at https://github.com/way0utwest, where I keep a lot of sample projects for things I work on with customers to demonstrate how to use Redgate tools, or just manage code better.
There are lots of ways to capture code, format it, and deploy it. However, you should use git and learn to manage your code within a team. I’d also suggest you use Flyway to deploy the code. There is an OSS version, and because it supports many platforms, if your company adopts PostgreSQL or DataBricks, you can still use a similar process to deploy code. Learn it and use it.
But first, get code into a Git repo.
One of the challenges many people have is focusing their learning efforts along some path. The best way to move forward is with steady effort that guides you through steps to build knowledge or skill. However, with so much information out there on the Internet, how do you decide where you focus your efforts?
Lots of people choose a random method, but the world is full of those people, many of whom never develop strong skills. That might be fine if you are an hobby guitarist or piano player, but it’s not the best way to approach your career.
Choosing what you want to learn is hard, but if you were to try and become a better Database Administrator, what do you think of this list: the ultimate checklist for Jr DBAs. This is a long list of topics in various areas, such as basic concepts, indexing, backups, security, objects, etc.
I think it’s not bad, but it leaves you a lot of work to do on your own. You have to find places that explain these items, which can be a chore. If you find a link, how do you know it’s good? You can’t judge because, well, you don’t know. You’re learning here.
I wish there were more guidance in posts like this. Not necessarily more information in this post, but with links that might help someone know where to look for good information. If you know these things, then what do you think is a good source of information? Maybe on your own blog or maybe an article that taught you something. I’m sure I don’t always provide background links, but I try to ensure I include links to help someone learn more.
Inside a company, I might include internal links to our policies or documentation. As an example, maybe we write down how often we think stats should be updated as a standard and why or what types of backup schedules are appropriate. This would be an easy way to help someone learn why we approach our jobs in certain ways, which is a very specific type of learning we need. Other more general learning is important and would drive these decisions.
Would you make a list like this one for yourself? If you’re a junior DBA, you might find this helpful, though I’d suggest you ask others for input on where you learn about these concepts. For people looking to learn something else, such as Fabric or Snowflake, is this the type of list that would be helpful, assuming there are some links on where to learn more? Let us know today.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
A customer recently was asking about grouping objects by type to see all the differences in two databases for one set of objects, like all stored procedures. This post shows how this works and what this changes for your system.
This is part of a series of posts on SQL Compare.
I have two databases, Compare_1_Source and Compare_2_Destination. I made a number of changes to the Source db and when I run SQL Compare between these, my default view shows me the different objects, as well as those in one database and not in the other. You can see this below.
This is the way most of us want to make changes, by deciding from this short list what changes to deploy. If you look closely, you can see there are 115 objects that are identical, so it’s nice to be able to see what has changed.
However, I have had a few people ask to see all the stored procs that have changed, so they can decide what needs to move. They aren’t ready for table changes.
In the upper right corner, above the destination database, I have a drop down for grouping options. I can use the default (type of difference), switch to type of object, or have no grouping.
If I choose object, I see this grouping. In here, the stored procedures are expanded, showing my differences. Second, in the middle, near where the checkboxes are for object selection, I see a count of how many objects are selected, and how many are changes. However, I don’t know the type of change.
I’ll modify a procedure and delete one and I see something slightly different. Now I see the object name to the right side of the checkbox, so I can infer a change if the name is on both sides, or a delete if there is an “x” next to the name.
My default view of tables is shown below. Note that my table changes are mixed within non-changed tables.
However, I can click on the Last modified column and resort the data. If I sort descending, then I see my table changes at the top.
Toggling these settings allows me to see different views. If I just want changes without groups or differences, then I can set no groups and see this (I’ve sorted by modified date).
I have a request to hide the unchanged objects, but that’s not something we do now, nor am I sure we will change things. You can submit your own ideas on Uservoice and get some votes from friends.
SQL Compare is an amazing tool that millions of users have enjoyed for 25 years. If you’ve never tried it, give it an eval today and see what you think.
This is part of a series on observability, a concept taking hold in modern software engineering.
One of the interesting things I saw in an engineering presentation on Observability from Chik-Fil-A was that they are sometimes bandwidth-constrained at remote sites. In an early version of their platform, they sent logs back to HQ, and their logs used all the available bandwidth, so they were unable to process credit card transactions.
While most of us don’t deal with lots of remote offices sending data back to a central data warehouse, we do often work in distributed environments, and we may send data to/from a cloud or even employees’ remote offices. Or maybe we send a lot of data between components. Bandwidth is very good in many parts of the world, but it isn’t infinite.
In the presentation, they talked about a tool, called Vector, that can work with lots of data, slice/dice/aggregate/sample/etc. the data, and then send the results to a sink location. This works like many other ETL tools that have a source and sink, along with various transforms that operate on the data.
It’s an interesting philosophy to try and send back metrics that might be useful to developers or Operations staff in understanding the performance of their system. By only sending metrics, the load on downstream systems is reduced. This also allows us to store less data and read metrics sooner rather than storing all the data and processing it each time someone needs a metric.
The flip side of this is that taking this approach means that the consumers of the metrics need to ensure they are getting useful and actionable information. Determining what is needed will be like any development project, something built, iterated, re-tested, and repeated. This might even be an ongoing part of building software as new features and logging are added to your software or system.
In general, I prefer to have more data over less, but the volumes of logging and instrumentation data have grown dramatically. Some systems are producing more log data than actual data on a daily basis. Like audit data, we likely need to reduce and limit the amount of data stored long-term. However, we want to keep the important data that we find useful.
I am looking forward to trying out Vector and seeing what’s possible. Having good CLI-based tools that can work with data is becoming more important all the time, especially as more of us move to DevOps flows, coding our systems operation in text, storing it in version control, and deploying on demand.
If you’ve used Vector, let us know what you think, and if you prefer another tool, share why today.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
hiddled – adj. feeling of loneliness of having to keep a secret to yourself.
I don’t know I am often hiddled. I don’t tend to keep secrets by myself. Often I do share with my wife, because it’s important to me to unload on someone, discuss something, or make them aware. That last one seems to be more important as I age.
That being said, if I didn’t share a secret, I would feel quite hiddled.
From the Dictionary of Obscure Sorrows
Thanks to everyone who attended, though with one stage I had a fairly captive audience. I enjoyed the event and look forward to coming back to the Twin Cities in the future.
A few nice questions, which I’ll address in a few posts:
- What are feature flags
- How do feature flags work in SQL?
- How do we reorder work when we’ve deployed to QA?
- Is there a way to avoid blame in a retrospective?
Slides: DevOpsDaysMN_BestPracticesForDatabaseDeployments
You can also watch the presentation on YouTube:
The third episode of Simple Talks is out. This is the new Redgate podcast from myself, Grant, Ryan, and Louis. The main page is here, and it has links to the audio versions as well as a the video one.
We recorded Episode 1 and 3 in Cambridge in June. Here’s a short behind the scenes look at the studio in the Redgate office.
We recorded a few more in Austin, which are being edited and scheduled. Look for those to come out in the next few months.
The final 2024 Redgate Summit in the US takes place in a few weeks, on Aug 21. Redgate Summit: The Database Landscape is coming to the Microsoft Office at Times Square in New York City and I’m excited to be going.
You can register today and join us for a full day of learning about DevOps, database development and management, and how to build better software. We have three tracks, each full of events for the day.
- New and Future Technologies
- Deep Dive Solutions
- Leadership
We have both Redgaters and industry experts on hand to deliver a wide range of sessions and panels. I’m not sure what I’m doing yet, but I’m sure I’ll be assigned a few of the TBD speaker slots.
Hopefully you can come enjoy a fun day in Manhattan in a few weeks. Tell your boss you want to learn how to better build and manage database software and head to the City. Then register and say hi to me in a few weeks.
Once again, I’m off to a conference next week. This time it is DevOps Days Minneapolis. I haven’t been to a DevOps Days event, though I always enjoy DevOps events because the attendees and speakers are very passionate about building better software faster. Some of them have great stories and experiences (and successes), while many also struggle with the same restrictions, priorities, and lack of attention to quality that most of us do.
In any case, this is a fairly small event, but I like Minneapolis and I’m looking forward to the trip.
If you’re coming to the event, say hi. If you can’t, there are lots of other DevOps days events coming up.
I visited a customer last week and attended SQL Saturday Baton Rouge 2024. Both were fun events, and an enjoyable week, though I was away from my wife for 4.5 days, which she didn’t love. Today, I had a quick turnaround, heading to Wisconsin Dells for THAT! Conference, which I attended last year and enjoyed. I didn’t submit this to the event, but got asked to go as part of my job. I accepted, and I’m gone from home for a week between these two trips.
I do travel a lot, but these trips got me thinking about how many of us might handle the unexpected demands from our companies. In this case, I had planned on SQL Saturday Baton Rouge, but not a customer visit. I got asked a month ago to add this in, which was fine, but two weeks ago the trip got extended by another day. Just before I got the update from that call, I agreed to go to THAT! for a quick presentation.
As a developer, I’ve been given last-minute work. It might have been a bug that was discovered or a new request that was high priority. In those cases, I’ve often had to work more hours for a short period, usually long days or even weekends. I’ve lost personal time because of the hours, but also the stress, as my mind is elsewhere. As an Operations person, I sometimes get stuck at work overnight or for long parts of many days.
In most of my jobs, I’ve been a strong performer and had a good relationship with management. I’ve almost always been able to negotiate comp time for the extra hours spent. In some cases, I’ve been paid for them, but usually, there is the equivalent of shorter days for a period or even missing some days of work. Often this is off-the-books as most HR systems don’t cope well with this.
For those of you who get unexpected demands, how do you handle things? Do you get something back from the company? Time, days, compensation, maybe even a thank you and a gift? It’s not something that I think is universal, but I’d like to think it’s common.
And if you’re at THAT, say hello to me this week. I don’t know when I’ll get some shorter days, but likely they’ll come in the next month as I try to catch up on ranch work.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
For my talk at SQL Saturday Baton Rouge 2024, here are the resources.
Slides: Architecting Zero Downtime.pptx
GitHub: https://github.com/way0utwest/ZeroDowntime
If you have questions, reach out.
vicarous – adj. curious to know what someone else would do if they were in your shoes, eager to watch another actor put their own spin on the character of YOU – carrying your body differently, speaking in a tone you never use, saying and doing things you didn’t even know were an option – a performance that might well end in disaster but would at least remind you that there are many different ways to play this role, even though you tend to assume you’re just reading the lines as written.
I am not an actor. I haven’t participated in theater or movies, though I have learned a bit about what it’s like from various presentations, promotions, and interviews I’ve done. While I’m always me, I don’t always feel as happy as I may appear.
That being said, I do think that I am just reading my lines by being me. My approach to life and work changes over time, somewhat like a rewrite of a character in a show, but these are the choices I make for a better life, career, relationship, whatever.
I don’t usually wonder how someone else might approach my life, though certainly I do wonder if others might perform better in things I do at work. Or coaching. I think about how another presenter might deliver my talk, or
I sometimes wish I were a better character in my personal life, but I don’t ever think about someone else would play my role. That feels too weird.
I mostly think about how I could learn to be better by watching someone else play their role.
From the Dictionary of Obscure Sorrows
I was trying to update my dbatools install to test something and go this error.
I fixed it with a little help.
The FixThe short answer from Chrissy LeMaire is to run this:
install-module dbatools -Force –SkipPublisherCheck
That worked.
Why?Chrissy explained this, but essentially it’s a certificate approval thing. Certificates work by verifying each other in a chain. One cert checks another, which checks another, etc.
In this case, Microsoft changed some things and while the dbatools goes through some of these things, the update-module is doing extra checks. Supposedly MS will fix this.
In any case, if you need an update, you need to bypass things for now.
Technology has dramatically changed the world over time. The advent of cars dramatically changed the US, as people could go places and meet others in a way that was difficult and slow before. The telephone let us communicate with people all over the world at a pace that was previously impossible. Computer technology has furthered this at a truly amazing pace, especially since the adoption of mobile devices by so many people. The flexibility in how we can integrate computer technology into our lives has been incredible.
However, each technology change brings about plenty of negatives and potential problems as well. I ran across a piece from L. M. Sacasas that has some questions we might ask about any technology, including the software we build. The start of the piece is that most of us don’t think about how our work might be misused, which can lead us to dismiss security risks or moral misuse risks. We often don’t consider the malicious ways people view applications.
The piece is interesting to read, but it ends with several questions that we might ask ourselves as we build something. I think a lot of these questions might not apply to our work with databases or corporate technology, but some do. I think many of them might apply if we think about the tools we use, especially AI.
Most things many of us build are re-hashes of something else. We might smooth the flow of work with better UX in an application. We might rewrite code to more efficiently use resources. We might implement features or reports in response to a business request, but we often are lightly evolving our software not changing it. We do, however, build things that others might misuse, either accidentally or maliciously.
The list of questions is interesting, but I also think we need to consider that others might use your system differently, not just from a UX perspective. Consider how they might exploit your software to achieve other aims. We may not be security experts, but others are experts and there are plenty of tools available to scan code and identify vulnerabilities. Use these tools with the knowledge that just because you wouldn’t use the software in a particular way doesn’t mean others won’t.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I’m headed back to Wisconsin Dells next week for THAT! Conference 2024. This is my second time in Wisconsin and third THAT overall. This time I didn’t submit, but Redgate is sponsoring the event and they asked me to deliver a session. I’ll be talking about Best Practices for Seamless Database Deployments on Tuesday morning, but there are a bunch of other good sessions taking place on Tue-Thur.
Each time I’ve gone, it’s been an enjoyable experience, learning about various developer and database technologies from others. It’s a fun family atmosphere and I recommend this conference as one to consider if you want to learn about any development topics.
Last year I learned about Rust and hummus in one talk, about containerized builds, and TDD. I also learned about Mind Over Milkshakes, which inspired that piece.
If you’re near Madison, this is a short drive. It’s worth checking out THAT and trying the conference if you want a family friendly developer event.
This post talks about adding custom metrics from the sqlmonitormetrics.com site automatically and how this works (and how it doesn’t).
This is part of a series of posts on Redgate Monitor. Click to see the other posts
What is SQL Monitor Metrics?SQLMonitorMetrics was a site that Redgate built years ago to hold various community submitted metrics that users of Redgate Monitor (formerly SQL Monitor) might care about or be interested in using. Myself, Grant, and a number of other MVPs and SQL Server experts submitted our own metrics in various categories.
At some point an integration was created to make the install of these metrics easy. This post shows how to do this and not to do this below.
How to Auto Install MetricsThe proper way to install metrics is to start in Redgate Monitor. Go to the config screen by clicking the gear in the upper right.
Next, in the Alerts section pick the custom metrics and alerts item. Click this.
This brings up a list of your custom metrics. In the text at the top, it says you can install a metric automatically, but you need to click the link my cursor is highlighted on below.
This brings me to the sqlmonitormetrics site. I’ll click find a metric. I could also click the Custom Metrics at the top.
I’ll select the first one for this post, but you can filter, search, or scroll around to find one.
The page for the custom metric has a button at the top. Please read the page and test the SQL to see what’s returned before you click this, but when you’re ready, click the button.
When I do this, I’m returned to Redgate Monitor, this time on the Create Custom Metric page, with the fields filled in. As with anything, be sure you review and check to be sure this is what you want.
That’s it. When you’re done, click create and your metric and/or alert will be set up.
How Not to Install MetricsThis works, but it’s a pain. If I go directly to https://sqlmonitormetrics.red-gate.com/ and then find a metric, such as the number of active backups and restores, I see this, an inactive button:
Because I didn’t start from Redgate Monitor, I can’t get this automatically.
If I scroll down, I see all the fields (in bold) and the entries. I can copy/paste all of these into the form, but this is a slow way to add metrics.
Don’t do this. Go to Redgate Monitor and start there (as shown above).
SummaryThis post shows how to automatically add custom metrics written by others from the sqlmonitormetrics website. This is a good way to add common checks to your system that others use.
If you see something missing, submit it, or ping me and I’ll write it or find someone else to do so.
Redgate Monitor is a world class monitoring solution for your database estate. Download a trial today and see how it can help you manage your estate more efficiently.
Today I have a question for you:
What are the three biggest challenges you face today as a database professional?
I was asked this recently, and I had some ideas, but I wonder if my challenges are the same ones you face. I don’t want to influence you, but I’m wondering what things cause you stress, headaches, difficulties, or somehow lower the enjoyment of your job.
Or maybe these are the things that challenge and motivate you to do more. It’s up to you to decide what is a challenge to you as a database professional.
I think about the challenges I have faced (and would face today) as someone working with data in a few ways. First, there are the actual challenges of working in an organization and dealing with other people/groups/management. Next, I think about technology challenges as well, with how we architect systems, pick tools, and make choices. Lastly, I also think about this in terms of my career and what difficulties I face there. How do I keep my career moving forward?
There are several ways to look at this. In a general sense with a high-level view or with specific challenges in dealing with technology/people/etc., such as HA latency. Maybe you think about long-term issues or acute firefighting that you are dealing with every week. There are many ways to view the impediments to an endless progression of smooth, quiet, easy workdays.
Spend a few minutes thinking about what challenges you have and let us know today what those are.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
One of the little details that I find matter more and more in enterprises is understanding why a tool behaves a certain way. OSS/home-grown ones often have limited docs, but vendor tools should have great docs. Today I learned about how to easily find Flyway comparison defaults, which is the topic of this post.
I’ve been working with Flyway Desktop for work more and more as we transition from older SSMS plugins to the standalone tool. This series looks at some tips I’ve gotten along the way.
The DefaultsDo you know all the SQL Compare defaults? Would you realize it if someone changed them? Do you think they’re the same in Flyway? I actually don’t know about the latter question, and I’m not digging through and comparing the options.
In any case, I saw an internal discussion recently about documentation and someone pointed out that we have this page that discusses where you change options. It’s good for that purpose, but it doesn’t list the defaults.
However, at the bottom, there are links:
- Oracle comparison options
- SQL Server comparison options
If you click through, then you get a list of default options. There is first a link to the full list of options (SQL Server, Oracle) as well as a link to SQL Compare options (SQL Server, Oracle).
The SQL Server page looks like this:
Note the sentence just below the image, which links to all the options. The second link is for the SQL Compare option explanations. Below this, you see the default options. This is a table of options, which looks like this image. Note this says certain options are set to true and all others false.
If I click through to the full list, I see this:
I can see all the settings and if they are required. I also see an example of the TOML file below this, where I can set these and store them in version control.
If you want to change behavior of the comparison engine in Flyway, or double check if someone else has changed something, this is the place to check.
Note, if you are looking to induce certain behavior, changes should be made in a TOML file for the project and flow through a PR process for approval and into a pipeline. Don’t edit these options directly, or change them in a pipeline.
Flyway EnterpriseTry Flyway Enterprise out today. If you haven’t worked with Flyway Desktop, download it today. There is a free version that organizes migrations and paid versions with many more features.
If you use Flyway Community, download Flyway Desktop and get a GUI for your migration scripts.
Video WalkthroughI made a quick video showing this as well. You can watch it below, or check out all the Flyway videos I’ve added:
I caught this piece on the need for programmers (developers) to not trust anyone, including themselves. It is written by a software developer for other software developers, but I think it can also apply to database work as well. It is a bit long, but it starts with the nature of abstractions in the world and how they let us work with simpler models of a situation or environment. However, most abstractions are leaky, and our assumptions about them can cause our systems to fail.
The leap from trust to abstractions seems a bit funny, but it makes some sense. We ought to simplify our situations so that we can generalize how to solve them, but we also need to verify things. There are a few examples, one of which is we ought to use tests to ensure the code does what we think it does, including using a wide variety of data. We ought to ensure that refactoring something doesn’t break the system, or more often for databases, we return the same results. Changing a query to run more efficiently with joins or a window function instead of a loop or subquery should return the same results. Not just for one row but all rows, and across different inputs.
One I especially like is the check on deployments. If we changed our code and had a deployment, did our code actually get deployed? In today’s world where different people might be responsible for merging and deploying code than those writing the software, we might want to verify that our changes actually got deployed. Perhaps reviewing deployment reports or logs can help ensure that we know the state of our live systems in addition to those in development.
There are some suggestions for how to become a better software developer, and as you might guess, this requires learning. I think in today’s world, some group exercises (katas) or reviews can be helpful as well. Maybe even practicing new techniques in a sandbox and running them through a PR process to let others see them. It can be scary to ask others to review your code when you are learning, but they might teach you something, or you might teach them something. We build in teams; we should think about learning in teams, at least periodically.
Becoming a successful software professional, data or application, takes regular work to improve our skills. Just as a woodworker might practice with their tools or a chef with recipes, we ought to practice with our tools. I’d hope that many organizations would also see this as something that needs to be encouraged with some amount of time allocated towards ensuring your staff continues to improve and produces high-quality results.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
solysium – n. the unhinged delirium of being alone for an extended period of time – feeling the hours stretch into days until a weird little culture begins to form inside your head, with its own superstitions and alternate histories and a half-mumbled dialect all your own – whose freewheeling absurdity feels oddly liberating but makes it that much harder to reacclimate to the structures and ambiguities of normal social life.
I don’t find myself alone too often. Not for an extended period of time. As an introvert, I like my alone time for sure, but I also like the energy of other people around. I just prefer some quiet times where no one is talking to me.
I don’t know that I develop solysium, with a culture in my head. I certainly game some interactions, or reply a conversation, usually wishing I’d said something smarter or wittier.
For me, I like the world, I like interacting with people and I appreciate the social structures I’m in. I am comfortable in a lot of US social situations, but I can find something fascinating or interesting when I’m in Italy or Germany or England or anywhere else. The differences, even if they’re not something I’d do or like, are amazing. They keep the world a special place.
From the Dictionary of Obscure Sorrows
I’ve been to quite a few of the SQL Saturday Baton Rouge events. There have been 10 with the 11th coming in a couple weeks. The crew down there has done a great job over the years, and I’ve met lots of friends there: Patrick LeBlanc, William Assaf, Kenny Neal, and more. Too many to list, but this has been one of my favorite events.
This year, SQL Saturday Baron Rouge 2024 is on July 27, and it’s once again at LSU. You can register today and join me for a fun day of learning inside, away from the heat. There is a packed schedule of all kinds of data related sessions. Learn about performance topics, Power BI, app dev, cloud, Data Ops, and more.
I used to fly to New Orleans each year and drive up. It’s an easy drive, and I even went to the Saints training camp one year. This year I’m going to Baton Rouge direct (well, I change planes) because I’ll be visiting a customer the day before.
However, if you don’t have plans Friday, there are two precons:
- PowerShell 101: Unleash the Power of Automation – James Petty
- Build an Outlier Detector in a Day – Kevin Feasel
These are great ways to get some fairly inexpensive training. Ping your boss and ask him to send you to one.
Register for SQL Saturday Baton Rouge and join me next week. Hope to see you there.
I’m in Austin today, ready for the Redgate Software 25th Birthday celebration. The company started in 1999 and this is their 25th birthday. All of our offices are celebrating, with most of us in the US coming to Austin for a party last night. My wife (and many partners) are here as well, emphasizing that we care about our people and recognize that partners support our employees.
I haven’t known Redgate for 25 years, but I have known one of the founders for 22 years. I first met Simon Galbraith in 2002 at the PASS Summit in Seattle. They were our first advertising customer at SQL Server Central in 2001 and the relationship continues through today. Redgate purchased SQL Server Central in 2006 and I’ve been working for them ever since.
In that time, it’s been interesting how my job has changed and evolved. I remember the early days of SQL Monitor being released and me demoing it from horseback.
There was the DBA in Space promotion, which I was only lightly a part of, but it was fun.
We had the
SQL in the City events, which included a tour around the US.
I still remember the first one in London at the Royal Society of Medicine in London. That was a treat.
and SELECT Star beer.
We ran that series for a long time, with quite a few live and virtual events. It was a fun time for my coworkers to get together in various places in the world. One of our last live events was in Cambridge at the Redgate office, with our 4 advocates.
Now we’ve evolved to the Redgate Summits, one of which is coming to New York next month.
I’ve watched Redgate grow to include Flyway (I’ve got a tips series), SQL Provision, and now Test Data Manager. I’ve done so many blogs and promos, the latest of which is TDM in 10 minutes. One of my favorites was a photo shoot in Cambridge, where they got me on another horse.
It’s been a great time for me and I’ve enjoyed my job with Redgate. I continue to do so today and look forward to the future. I truly hope this is the last job I have.
The cloud has been a controversial concept for much of its existence. While the idea has been around for many decades, AWS started selling IT services in 2006, with Azure following suit in 2008. Since then, the use of cloud services has grown tremendously. While some applications and organizations have embraced the idea from the beginning. I found many of you at SQL Server Central were very hesitant at first. I guess some of you are still skeptical about the value of a production database in a public cloud.
From the beginning, I’ve felt that cloud computing has a place in the world, but in a way that is more appropriate for some situations than others. In terms of database (and maybe compute services), if you have a very well-known and predictable workload, the cloud can be very expensive. It might still be a good choice, but I think it often isn’t. If you have a variable or growing workload, then the cloud might serve you better than trying to keep up with new hardware in your own data center.
Bluesky has had a tremendous amount of growth since its founding. Twitter invested in this as a distributed project and when Elon Musk purchased the site, many users moved away. A lot of them went to Bluesky, which had to deal with a quickly changing workload. They started in AWS, but eventually decided to move to an on-premises setup.
Why? One would think their continued growth would mean AWS (or another cloud) would be a natural fit. However, they hired someone that provided an analysis showing they could invest in their own hardware, overprovision what they needed for growth, and keep up with the demands as they had developed a fairly accurate method of forecasting future needs. The savings in purchasing their own hardware allowed them to buy more than they needed and handle short-term spikes.
To be clear, this doesn’t mean the cloud is worse for most or even many organizations. Bluesky knows they need to continue to invest in hardware, and they are prepared to keep adding resources. They also architected a distributed system that still allows them to scale into AWS if needed in the short term. I don’t know many organizations that would prioritize those things alongside the rest of their business. Most of us do a poor job of forecasting load. Even if we do, often the difficulties of purchasing new resources mean that we can struggle to meet increased demands.
The companies that have moved to the cloud with success, and those that have left the cloud with success, are those that measure, monitor, and make appropriate decisions based on operational data, not opinions and feelings. They aren’t afraid to make a decision one way or the other, choosing what’s best for the organization, not what someone wants to do or thinks will be better.
The cloud might be better for you, or it might be worse, but you ought to have a way to measure and analyze the options. You also need a talented staff that isn’t afraid to try new things and adapt their architecture to take advantage of modern hardware and software. Too many of us aren’t as flexible as Bluesky and might not have the success they have, in or out of the cloud.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
This was an interesting error, and I was able to duplicate it, so I decided to write a post on how to find the problem and fix it. The error after running DBCC CLONEDATABASE is:
NO\_STATISTICS and NO\_QUERYSTORE options turned ON as part of VERIFY\_CLONE.Database cloning for 'atest' has started with target as 'aSmallTest'.Msg 2601, Level 14, State 1, Line 11Cannot insert duplicate key row in object 'sys.sysschobjs' with unique index 'clst'. The duplicate key value is (885578193).
The final key value (885578193) for you might be different, but the error is the same.
Note: In SQL Server 2022 RTM + GDR, this error occurs with system objects collisions. Upgrading to CU12 fixed this. Possibly earlier CUs fix it, but that’s all I’ve tested.
The ScenarioI connected to a SQL Server instance and ran this:
DBCC CLONEDATABASE(aTest, aSmallTest) WITH VERIFY_CLONEDB;
I was just trying to copy a database to do some testing against a copy. The command too quite a few seconds (11 for me) to run before returning the error above. You can see the screenshot below.
Strange. Why would a copy of a database cause an error here? I’ve run DBCC CLONEDATABASE on this instance before and it worked.
I’m not sure of the exact problem, and my searches note that
The FixI found a post that describes a similar issue, but certainly isn’t the case here. Another post from Pinal shows how to query sys.sysschoobhs, which isn’t reachable with a DAC connection. I finally found in the docs that SQL Server doesn’t support cloning with objects in the model database.
So, I need to delete objects in the model database. In my case, I took this query (from the first link above) and ran it from the source database. That’s important. Running from anywhere else doesn’t work.
SELECT m.id, m.name, c.name, c.id, m.typeFROM model.sys.sysobjects mFULL OUTER JOIN sys.sysobjects cON m.id = c.idJOIN sys.objects oON c.id = o.object\_idWHERE --o.is\_ms\_shipped <> 1m.name <> c.nameAND m.id IS NOT NULL;
As you can see below, this returns two objects.
If I look in model, I see these, one if you just look at tables, but the PK is attached.
If I delete these two objects, then DBCC CLONEDATABASE works.
SummaryThis is a strange error, and I’m not sure why it appears, but the documentation notes that running dbcc clonedatabase with objects in model is not supported. I suspect this is a change across one of the CUs, as I know this used to work.
In any case, the fix is remove the objects in model. If you really need these, then I’d create a script to remove and add those objects back, with a call to dbcc clonedatabase in the middle.
I assume that most of you know about the principle of least privilege. If not, please read this short blog from Brian Kelley and make sure you understand how you should approach security. In the modern world, we also ought to adapt our systems for the zero trust model, which includes the least privilege principle.
However, I wonder how many of your organizations really follow these security guidelines internally. Are you strict about adding limited access and removing it when people change jobs/roles? If you use Windows Auth (or Entra), are your admins doing that or just adding in new roles? Do you scope down database access roles in granular ways or just stick with 1-2 roles for the most common things people do?
Maybe more importantly, do you use roles or are these systems that still have explicit grants for users?
Microsoft had a major hack recently from a test account that had administrative privileges. While there certainly might be a need for a test account to have privileged access, I’d hope that any test account created had a limited lifetime. I’ve created privileged database access accounts for vendors, but usually set a reminder to myself to disable the account after xx days. When I got smarter, I wrote a one-time job to do that and scheduled it. These days, I’d also file a ticket for my team noting that this needs disabling as well.
Humans get lazy and often don’t think about the future. If you’ve never had an issue with a test account, why think something might happen? Why spend the time writing a note or a job when surely you’ll remember or deal with it later? Maybe more common, why disable a login when the user might need access longer? We don’t want to deal with another phone call and enabling the account. That’s an interruption to our work week.
What has been humorous to me is that I’ve seen quite a few people who are very security conscious get annoyed when some automated system or process disables their account and forces them to make a call.
It is annoying. However, these little things, the details, the adherence to good practices are what help ensure we have better security. When we take shortcuts (like not enabling MFA), when we skip steps, when we do small favors for others, we’re increasing risk. Most of the time that’s fine.
Once in awhile it really comes back to cause problems. I’m not sure the savings are worth it.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
This isn’t data related, but I had some fun, so I decided to make a quick post. I’m on holiday today, actually yesterday and today, in San Francisco/ Oakland.
The reason: a date.
My wife is a fan of Dwele and she’s often said that since he lives in Washington D.C. and is sometimes in clubs there, we should plan a trip sometime. I’ve periodically searched for concert dates and never seen any. While I was traveling this spring, I happened to see a couple dates at Yoshis, a jazz club in Oakland. I bought tickets and surprised her. I then booked a short holiday this week for us to fly out and see the concert.
Tonight is the show and we’re excited for the adventure.
Life is short. Work hard, but remember to enjoy yourself when you can. Especially with those you love. Look for opportunities to bring them joy and take them when you can.
fitching – v. intr. compulsively turning away from works of art you find frustratingly, nauseatingly good – wanting to shut off the film and leave the theater, or devour a book only in maddeningly little chunks – because it resonates are precisely the right frequency to rattle you to your core, which makes it mildly uncomfortable to be yourself.
Good is relative, I guess. Maybe things are so well done that you can’t handle the imagery?
I’ve found myself fitching while trying to watch Crash many years ago at home. My wife was captivated, but I had to get up and leave the room. It was far too real, and too possible for me, and I couldn’t handle it.
That has stuck with me, and I’ve avoided certain films or books since then.
From the Dictionary of Obscure Sorrows
Stretch Database is finally going away. It is being retired. It was deprecated on Nov 16, 2022, from SQL Server 2022. Effective Jul 9, 2024, the supporting Azure service is retired. I saw this in an announcement on Jul 3, though I hope anyone using this service has been seeing lots of reminders over the last couple of years. I know I’m getting MySQL retirement notices for one of my services and need to migrate some workloads this month.
If you tried this service, you might have realized that the pricing didn’t make sense for most of us. If you hadn’t tried it, it worked by moving some of the data in your tables into Azure, where it could be queried if needed. It was an interesting idea, though most of us would have wanted this to work between two SQL Server instances, not between SQL Server on-premises and Azure.
In any case, if you’re on an older version of SQL Server, the recommendation is that you bring your data back on-premises. If you have SQL Server 2022, they recommend CETaS (Create External Table as Select), which lets you query data in Azure storage. This lets you put some data in text formats and query it as needed, reducing the use of expensive relational storage disks. Parquet is the recommended format here. There’s also a weird mention of Fabric in the announcement, which doesn’t seem to fit with the objective of the rest of article.
I haven’t worked with this enough to know how well this performs or what patterns might fit here. I do know that reducing the queries from clients, especially SELECT * and unbounded queries across all your data helps. If your clients always want to query all data in a table, then nothing works well. Building systems that query hot data by default, and filter away from warm/cold data is always helpful. Having some good indexing is also important.
I don’t know of anyone that uses Stretch Database, so I’m not sure how many people are affected here, but I hope they knew about this before Jul 9.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Test Data Manager (TDM) is a suite of products from Redgate that make it easy to build dev and test databases in seconds. It’s a nice rewrite of a number of pieces of technology that we have sold for years, and it was launched at the PASS Data Community Summit in 2023.
I’ve been working with a few customers and sales engineers as they evaluate the fit for TDM in their environment. One of the challenges we’ve found is that the setup can be complex, and the knowledge required to get up to speed is high. There are a lot of moving parts to get this working in a way that makes it seem simple for the end users (usually developers). As a result, one of our engineers, Alex Yates, built a mini-PoC system using PowerShell that’s available.
This post shows how you can get started to demo subsetting and masking in less than 10 minutes.
Getting StartedThere are really three things you need:
- dbatools
- Redgate tools (and a Redgate account)
- git
- make sure c:\windows\temp exists
Get the first and third ones from the Internet. For the second, you need to get these tools from your account executive, or just start a trial. The tool will do that. These tools run on various platforms, so ping your rep or sales@red-gate.com.
Next, get the repo from here: https://github.com/alex-yates-redgate/TDM-AutoMasklet
Git makes this super simple. Just clone this down.
ConfigurationThere isn’t much to configure. In the repo, open run-auto-masklet.ps1 and look at the first 15 lines. These are where you might change things.
The local instance is set in line 2. If you have a named instance, use that. I’ve included a config file below that I used on a named instance.
That’s it. The repo includes a copy of Northwind in an install script to create the full sized database as Northwind_FullRestore. The subsetter will then move a portion of data to Northwind_Subset. If you want to change these names, you can do that.
Running the ToolThis tool can run run as a normal user, but if you need dbatools installed (it will do that), then it needs to run as Admin. I added that as a requirement above, so you don’t need to run this as an admin.
Here’s what the tool does:
- Get dbatools
- Get the latest versions of the subsetter and anonymize.
- authorize you, and start a trial if a license isn’t assigned to your Redgate ID.
- drop the two databases (Northwind_FullRestore and Northwind_subset by default)
- Creates the two databases with schema (and data for the full restore)
- pauses with output
- runs the subsetter to move a portion of data to the Northwind_Subset database
- pauses
- runs the classification process against Northwind_Subset to classify columns
- pauses
- Runs the masked against Northwind_Subset to mask data
Here’s the first set of output, showing the config and first part of the process.
Here is the first pause. You can see there are db create notes and then an explanation of what to see:
I like that this gives the subset command, which takes some getting used to. The TDM GUI hides this, but every customer has wanted to customize things, so this is a helpful way to do the PoC.
The subsetter does a lot, as you can see below, but basically it map out the database and then starts to determine which data needs to move. In this case, lines 10 and 11 of the source scripts shows that we are subsetting dbo.Orders with the OrderID<10260.
When this is complete, we get another message that explains what happened. We get some telemetry as well with the time taken here.
We also see the next part of the process, which is classifying the data. Again, we see the command for this, and you see this runs quickly.
Lastly, the next pause tells us there is a classification file at a particular location. We get the path if we want to look or edit the file.
Then the masker runs, and we see that 5 tables are masked. We get telemetry and below the results you see, there is more info on what’s happened and what to look for in the databases.
Checking the ToolOnce the execution is complete, I decided to look at the two databases. In SSMS, I had a vertical tab group to compare things.
First, subsets. I’ll count orders, order details, products, and employees. You can see the original db on the left and the subset on the right. Less data.
Not super impressive, but imagine there were a factor of 1000 on the left. That would be cool.
What about masking? Let’s check.
The Shippers, Suppliers, Employees, Customers, and Ordere tables were masked. Let’s look at Shippers. We can see the phone number is masked.
Checking Employees, I see less as there is a subset here, but I see data masked.
You should see similar results, and what’s more, you can alter the various config files or filters to test how your changes work.
I’m a big believer in sandboxes for learning and experimenting. This gives you a nice sandbox. You can change the various files or script and then re-run the tool in a couple minutes to see your changes.
Here are the CLI docs you might use to change things:
- Classification – https://documentation.red-gate.com/testdatamanager/command-line-interface-cli/anonymization/classification
- Masking – https://documentation.red-gate.com/testdatamanager/command-line-interface-cli/anonymization/masking
- Subsetting – https://documentation.red-gate.com/testdatamanager/command-line-interface-cli/subsetting/subsetting-configuration/subsetting-configuration-file
Give it a go and see what you think.
If you want to see a video version of this, check this out:
I almost missed this month, so this is also a good #SQLNewBlogger post. I thought about it for a few minutes as I ate breakfast at my desk and then knocked this out.
This is the monthly T-SQL Tuesday blog party. I manage the site at tsqltuesday.com, trying to keep the party going. I have a lot of help from hosts each month running the topic, and I appreciate their efforts. Join in an write, and then host a month. Lots of people have done it.
Past AdviceI struggle with this, as I’ve had a great life. I wouldn’t change anything, given where I am today, 33 years after my first data job. However, knowing I’m not changing my life in some time travel way, this is something I wish I’d have known about in my early 20s.
Network and help others in the community.
I’ve done this lightly in my first few jobs, but mostly within organizations. As I’ve grown and changed jobs, I’ve seen tremendous power in people getting together to talk, to get to know each other, to share problems and solutions, to present their knowledge and learnings with others.
User groups were a core part of this, and out of them grew the PASS organization and Summit, the SQL Saturdays, and the amazing community we have. It’ s far different than other communities, and many of them see it as well. They wish their world was like the data platform world.
The power of networking is amazing. The rich world that comes from community is something special.
It’s still there today and it’s worth joining, no matter where you are in your career.
I had a lot of local branches for a repo (actually a few repos). I know these are old and not used anymore, so how do I delete them? This post shows how to do that on Windows.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. You can see all posts on Git as well.
The ProblemAs I’ve been making changes for various SQL Saturday events
I saw this SO post, which was a good starting point. I grabbed this code, which I’ll explain below.
git fetch -p && git branch -vv | awk ‘/: gone]/{print $1}’ | xargs git branch -d
The problem is this doesn’t work on Windows.
Running This on WindowsI assume most of you installed Git and have Git Bash. The xargs and awk commands are Unix/Linux ones, so you need a bask shell to tun them
The solution for me, was to open a bash shell in the repo with the right click menu on Windows.
Then run the code:
Local branches removed. Well, almost; read the next section.
AdditionsNote that in the first execution, I had two errors noting that there were some unmerged branches. When I look at these, I see they were old branches, ones that haven’t been used in years. I’m guessing either I was fixing something for someone, or they fixed something in another branch.
So, I forced delete by re-running the command a capital D.
How this WorksThis code uses some Unix based utilities that I haven’t used in a long time. The flow of this is similar to how PowerShell, or even VBScript works, but on a single line. In this case, this code:
- Gets a list of branches from the remote with git fetch after pruning the references for local branches that don’t exist on te remote.
- Run the branch command with the verbose output. Could be –verbose as well
- Take the output if the previous step and pipe that through awk. This command will parse text, looking for “gone” in a line and then printing the branch name.
- This text is then taking with xargs and passing it to the git branch command with the delete option.
Note this doesn’t force delete branches.
SQL New BloggerThis post took about 20 minutes to write. I spent about 5 minutes checking a few code examples online, and then tried one after I’d killed branches from GitHub. I don’t have a great solution there, but I don’t do this often and I can click a few buttons to manage this.
I then structured this post with a few screenshots and spent 15 minutes working on it. I’d actually sketched it in 5 minutes with the major sections and a sentence in each and realized this would be quick to write, so I just filled it in on a Sunday morning.
You could do this as well and give an interviewer something to ask you in the next interview. This might catch their eye. I’d also suggest (and I will) do a few posts on awk and xargs. Those are good skills to have and you might spent 20 minutes experimenting and having fun with them.
I saw a blog post from Randolph West recently that asked How do you restore a SQL Server 2000 database in the year 2024? It’s a bit of a process, involving an intermediate version and two restores. He also points out the need to run DBCC after the first restore, which is a good idea. I wonder how many people would take the time to do this, or even think about it as an upgrade step?
This was interesting to read as I had a customer ask me about doing this a few months back. They were trying to clean up their database estate and modernize some of their older systems. This was becoming a big project for them, as they had several pre-2017 systems, none of which were in support. Auditors, regulatory authorities, and even business partners see this as a large security risk and get concerned if you’re running older software.
I’ve felt that in most cases, I ought to be able to run a database server for close to a decade. I certainly need to patch it with CUs in that time, but the support lifecycle says that you get mainstream support for 5 years and then extended support (paid) for 5 more. That extended cycle also includes security patches, so ten years seems reasonable.
As a side note, the final support lifecycle for 2014 ends on 9 Jul 2024. That’s a decade if you upgraded in the first year of release.
However, many of us have multiple instances, and upgrading those can be a chore. Perhaps you trust that nothing breaks, but I would say for many larger organizations, upgrades are a constant fact of life, and it is important to probably start testing upgrades at five years, knowing it might take 1-2 years to upgrade all instances of a given version. That’s if you don’t find issues in testing. If you test a 2017->2022 upgrade now and find issues, you might spend time mitigating these, or maybe wait for SQL Server 2025 (my guess) and hope you don’t have the same issues. There are also the challenges of in-place vs. side-by-side upgrades, and you might choose one in testing, but decide to change for the final upgrade for various reasons. All those things can cause delays.
I still find myself a little nervous about the “evergreen” versions of SQL Server, where Microsoft patches them as needed. I know they try hard not to break any backward compatibility, but if they do, then you’re stuck. I prefer to schedule my upgrades and make them a normal part of the DBA job. That being said, don’t drag them out for years and years. If you still have SQL Server 2012 or older versions, you’re doing something wrong.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
One very common targets for hackers are schools and universities. The latter may have plenty of resources for staff, but often school systems don’t have that same type of budget. In many ways, it’s sad that hackers would target schools that struggle to educate and help others. The staff often deals with low pay and high stress already, and losing access to computer systems adds to an already tough job.
There was an interview this week with the director of technology at one district. Johnathan Kim works for the Woodland Hills school district and is a former staffer at the Navy Cyber Defense Operations Command. That’s the type of training that I think few school district employees have. I’ve known a few people who manage technology inside schools, and while they are often smart, capable people, they aren’t security professionals.
The interview talks about a few of the changes that Mr. Kim has made, such as enabling two-factor authentication (2FA) and removing local admin rights for many teachers. I know these are the types of rules that frustrate many workers who use computers. In fact, I ran into someone who rarely upgrades software on their development machine because so many applications require administrative rights and they don’t want to bother opening tickets more than a few times a year.
Two-factor authentication can be a pain, and I know I get confused sometimes as I have both 2FA and MFA with different processes for different systems. It’s good in that a few times in the last year I’ve caught a hacking attempt, but it’s also a pain to deal with when I’m doing something simple. I can see why people don’t like it when they don’t understand the challenges of securing systems. Every time I find myself frustrated, I stop and remember the problems others have had, especially those that have dealt with ransomware in their organizations.
A good point in the interview is that education can help smooth the way for security practices that feel unnecessary or disruptive. It’s good to remind ourselves why we want the least privileges needed assigned to others, and why those of us with privileged access need a second account for that access. We also ought to come up with a good story to educate others when they complain, perhaps using a story of a breach or loss to help remind others that our systems are constantly under attack.
Steve Jones
Note: I DO NOT recommend this. Any changes to a pipeline should be in code and through a PR.
That being said, I know this information is out there and some people need it. The question from a friend was how can they set a variable in an Azure DevOps Pipeline at runtime. This was for testing, and they wanted to change the pipeline behavior to test things when they ran them.
This post will show how to do this in classic and YAML pipelines. As a scenario, I’m just going to get a directory listing of a folder, and change that at runtime.
Classic PipelinesI know the trend is everything in code. For experimenting and learning, I find this slightly annoying, so I like classic pipelines. I know others do.
In a classic pipeline, I can set variables. I’ll add a new one and call it myLocation. Over on the far right, there is a checkbox for “settable at runtime”. Check that.
Now, I’ll add a task to this pipeline that runs a dir, using this variable.
I can save and run this, and I see the results of c:\Users from a hosted agent.
That’s the default behavior.
Now, let’s alter this at runtime. When I click “run pipeline, I see this on the right side as a blade. Note the “variables” section below.
I can click this and see my variables. System.debug is set at runtime by default, but I see my other one.
If I click this, I can change the location. I’ll set this to c:.
When I let this run, note I get different results.
I’ve changed behavior at runtime.
YAML PipelinesIn a YAML pipeline, I don’t have tabs or variables. Instead, I just get a script of sections, like this.
I can alter this to add a variable by looking in the upper right, where I see a “variables” button. Click this.
I get a list of variables, which is none in this case. I’ll click “New variable”.
This gives me a dialog where I can enter the information. Note I can set a default as well as let users override this with a checkbox.
When I save this, I see my variable.
Now, I can alter my script. I’ll add this as $(myLocation), where I surround the variable name with a $ and ().
I can validate and save this, which I do.
It’s valid, because I typed well, but this really should go through a PR. Since I’m testing, and I’d approve the PR, I’m doing it in main. I shouldn’t do this in any org.
Now when I run the pipeline, I have the variables item where I can change the variable.
I can also set this variable in YAML, like this:
However, if I set that value, I can’t change this at runtime. Here’s the runtime screen.
I can use a parameter instead. I’ll use this structure:
When I run this, I see a new box:
I can override this “Dir location”. When I set this to c:\, I see these results:
SummaryI’ve shown how to configure a variable to be set at runtime, both in classic and YAML pipelines.
Note, this a place an administrator can make a mistake, or run rogue, without review. This is not recommended. Put all pipeline changes through a PR.
Yesterday was Independence Day in the US and a day off for me. Today, I’m back to work. Same for my rocket engineer son, one day off. My wife remembers a few of her companies giving a 4-day weekend when a holiday falls on a Tuesday or Thursday, but that hasn’t been my experience and isn’t this week.
I know we’re barely into summer, which technically started Jun 21, but most of my life has measured summer between the end of one school year and the beginning of another. That’s usually late May (Memorial Day) to early September (Labor Day). I think that’s a very American thing, as I know lots of my friends in Europe are just starting their break between two grade years. For those of you in the Southern Hemisphere, I’m sure this time of year is even more different.
In any case, since it feels like midsummer to me, I wanted to take a break from technology and write about interesting fun things I’ve seen lately in the world of books and tv/movies. Even as I write that it seems strange since I think TV isn’t a concept for me anymore. I’ve gone completely to digital streaming, with episodic shows that aren’t ever broadcasted. They just drop, individually or as a whole season at once.
In any case, I’m starting with Presumed Innocent. I read this book a long time ago, probably in 1987 or 1988. I couldn’t remember the story, but a friend recommended the series on Apple+. The first episode started slow, and I wasn’t sure I’d watch more, but the end caught both my wife’s and my own eyes. We had to watch the second episode right away. It’s dropping episodes week by week, which feels very old-fashioned and unsatisfying.
I’ve also been working through Lost in Space on Netflix. I’ve watched the various versions of this since the old black-and-white TV version. I went through Season 1 of this a few years ago and really enjoyed it. I just realized there were more seasons and have been enjoying them. It’s a bit silly, far-fetched, and inconsistent in places, but still entertaining as a sci-fi view of colonists traveling through strange places in the galaxy.
My wife and I completed season 1 of Resident Alien on Netflix as well. The casting is great and it was very entertaining. A fun watch for us. I haven’t found a lot of funny series lately, and fall back on old episodes of Scrubs, Community (first 2 seasons), Brooklyn Nine-Nine, the US Office, and Mom. Even though I’ve seen most of these series, they’re still good for a laugh. Once in awhile I even see an episode I haven’t seen (or don’t remember). In any case, Resident Alien was good.
I like to read a lot. This summer, I’ve caught up on the latest in a few series I’ve enjoyed over the years. I’m working on Spinward Fringe 17, along with Observability Engineering. That latter isn’t entertainment, but it is interesting. I’ve also enjoyed these books lately: Toxic Prey, I Will Find You, Murder One, The Last Detective, and The Unincorporated Future. I’ve read a bunch more, but those were the highlights.
I haven’t see many movies this year, but I have watched a few documentaries. The Thriller 40th Anniversary, the We Are the World documentary, and the George Michael one were great. I did see One Love (on Bob Marley), but it wasn’t that good. The music and actor are good, but the story isn’t. If you want a fun, unusual watch, try the Donut King.
If you’ve got some fun escapes from work, either written or video, let me know. I enjoy watching a Slack channel at work that is dedicated to movies and tv, getting some recommendations there. I’m sure I’ll see something from one of you that’s worth checking out.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
symptomania – n. the fantasy that there’s some elaborate diagnosis out there that neatly captures the kind of person you are, tying together your many flaws and contradictions into a single theme – which wouldn’t necessarily sort out the mess inside your head but would at least let you mark it with a little sign so people know to walk around it.
Another great definition. Maybe we should have signs we can hold up to let people know we’re feeling a little off and they should just walk around us and ignore us
I don’t feel symptomania, mostly because I am accepting of flaws, and I don’t try to simplify the world so much anymore. I don’t see black and white but rather lots of gray. I don’t try to reduce things to a simple measure, but accept a complex way of the world as the way things are.
From the Dictionary of Obscure Sorrows
I had a customer that was looking to document a restore that had occurred on one of their systems and didn’t see it. They had concerns about SQL Server accurately tracking history across time and noted they hadn’t cleaned any history.
We dug through some of their instance jobs and found one that ran sp_delete_backuphistory. The person didn’t realize this removes restore history as well. This post talks a bit about how this works.
The important thing to understand here is that this removes backup and restore history. Not just backups. I don’t know I like this, but it is what is documented (emphasis mine).
In this case, the sysadmin didn’t realize this removed restore entries. Once they did, they stopped worrying about things. We could have potentially restored an old backup of msdb and found this data, but they elected not to do this.
How The Procedure WorksWe can actually see the code for this proc. I have expanded the msdb programmability section under system stored procedures.
I won’t show it, but this works in the following way:
- create three table variables with a single ID column
- insert data into these two tables from backupset where the date is older than the parameter passed in.
- backup_set_id from backupset
- media_set_id from backupset
- insert data into the third table that matches the backup_set_id from the table in A
- start a transaction
- delete from backupfile the matching backup_set_id values
- delete from backupfilegroup the matching backup_set_id values
- delete from restorefile the matching backup_set_id values
- delete from restorefilegroup the matching backup_set_id values
- delete from restorehistory the matching backup_set_id values
- delete from backupset the matching backup_set_id values
- delete from backupmediafamily where the media_set_id values match
- delete from backupmediaset where the media_set_id values match
- commit the transaction (or rollback if errors).
This is a pretty simple flow, and it works well. The tricky part is that the is joins data in a way that makes sense, but might not be what you expect. This doesn’t remove restores based on the date, but based on the backup rows being removed.
Know Your ToolsThis is a poorly named procedure, but that’s not an excuse for anyone. If you use this, and likely should, you need to ensure that you understand how it works. The phrasing in the documentation makes sense, but it can be a little misleading as many of us might assume the date is applied to backup and restore history tables.
It is not.
I had someone ask me recently how to run xp_cmdshell on a Linux version of SQL Server. I told them you can’t, as it’s an unsupported feature and not one that I expect to see released. I had to double-check, since I did think that supporting a BASH shell was a possibility, but it wasn’t added to the product.
In the feedback forums, I saw a request for xp_powershell, though the feedback from MS is to use CLR for this. They suggest external access permissions, but those aren’t supported on Linux. I also didn’t see a request for shell scripts added, and I’m not sure I want one.
A few years ago, I wrote a piece on the dangers of xp_cmdshell, as this does create a security risk. I can be mitigated, but the modern world is complex and it can be easy to make a mistake here. I’ve used xp_cmdshell often without issues, but I’ve also known the risks and tried to mitigate them with controls on the machine, network, and who can execute the procedure.
I’ve seen people use xp_cmdshell for a number of tasks, like exporting a result set to a file, checking disk space, moving files after a BULK INSERT or backup, or some other task that is tightly related to actions taking in T-SQL. This can be a very handy utility for many administrators.
Today I’m curious what are your use cases. Where do you use this utility, or where is it much easier than adding a PowerShell step in an Agent job? Similarly, do you use xp_fileexist or other XPs to do things are are outside of the realm of T-SQL. Leave a comment below and let us know how you use this stored procedure. Or in which situations this has proven to be useful in the past.
And maybe vote for a BULK EXPORT
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I wrote an article that was published on SQL Server Central on how to get your scripts into Git. This post adds a few more thoughts on how you might get started.
This is part of my series on git that is designed to help people get started using version control in their daily work. You can see all my other posts on Git as well.
Organizing My ScriptsI showed a sample folder that looked like this, with a main area of scripts and then a subfolder for reports.
As a DBA team, I want to ensure we all easily can find things. If we don’t have a lot of scripts, I might keep most in a single folder. However, if this becomes 3 or 40 scripts, it’s easy to make mistakes or have too many similar things.
What I might organize things slightly better like this:
Here I’ve moved the Diagnostic queries from Glenn Berry into their own folder. Those I might run more rarely, though I might update them more often. Getting them into their own folder lets me move them out of the way. You could organize those by version, but the names keep them separate, so I’d probably just keep them all there.
I added a “ETL” folder for specific scripts related to that function. I might need those regularly, but this helps me find them. If I had different types of ETL stuff, like on-prem and AWS, or maybe different apps (“Sales DW” vs. “Inventory ETL”), I might put those in subfolders below there.
I also renamed the who_is_active.sql to “common_who_is_active_scenarios” where I have some calls with specific parameters set.
I didn’t do this, but looking at this, I’d probably add a “installation scripts” folder where I moved the sp_WhoIsActive.sql and other install versions of scripts into that location.
What I’m trying to do is just get DBAs to easily and quickly find scripts without accidentally picking the wrong scripts. This helps in pressure situations and also helps onboard new team members.
I didn’t show how to update and version scripts, but I’ll do that in a new SSC article.
I often deal with customers who are looking to improve the way they build and manage database software. These could be small companies or large enterprises, with teams of developers trying to enhance their application software to solve new business problems. Often those enhancements require new data, with the related schema changes in a database. Even if there isn’t any new data, often we need to query data in new ways, combing, filtering, aggregating, and otherwise transforming data into extra information that a business can use.
Solving many of these problems is iterative by nature. We try one thing, then another. Often a developer might experiment with a data model or query, trying to match a requirement they’ve been given. Once they produce a solution, we may find problems in testing, or far too regularly, in production. These could be data-related issues, where the developer hasn’t considered values in their solution (zeros, blank or long strings, extreme dates, etc.). These could be logical errors, where the developer just made a mistake. There could also be a problem with the requirement, where the customer provided an incomplete or incorrect specification to the developer.
In any of these cases, what often happens is rework. We need to re-look at the code we’ve written and change it in some way. This isn’t a complete rewrite, but looking for the logical error, incomplete algorithm, or poorly performing structures. Then we need to (test it and ) move the code back through our process of deploying to QA, UAT, production and anywhere else the problem exists. The further we are in the process (closer to production) the more costly the effort to rework code.
Some rework is unavoidable, but I think quite a bit isn’t. If we wrote better-performing and cleaner code the first time and had a pattern for testing our code, we might avoid a fair amount of rework. If we caught silly mistakes early, we could get more done.
I think that often developers minimize the effort required to perform rework. They think small changes are easy. After all, we already know the problem space, we’ve looked at it, and the fix should be quicker than the initial analysis and development.
Or is it?
One of the things I’ve seen is that everyone is busy. We are pulled in many directions by many different people and work on lots of different problems. Even when we work on the same application every week, it takes some time to remember the context under which we wrote the code. If it’s not our code, then we spend time trying to understand how the original developer approached the problem. Without good specifications or tests, we might not view things the same way during rework. We often fix one problem but create another.
Automated testing should help here, as the tests should codify the understanding of the requirements, but too few database developers use automated testing for their code. Even developers who embrace testing from the application might not have sufficient test data to ensure their code will work in production.
Most of us are expected to do more work this year than last year. We’re constantly battling a lack of resources. We can’t necessarily write more code, but we can learn to be more efficient. We can learn to reduce our rework by writing code better the first time. That takes some effort and knowledge. It takes working as a team, sharing successes and struggles, and adapting our code to use patterns and avoid anti-patterns. It requires some automation to embrace static code analysis and testing to avoid silly mistakes getting deployed and wasting time. More importantly, it takes support from management to spend time learning as a team to produce better code and avoid silly mistakes.
A little investment in learning and practice goes a long way. Unfortunately, too many people, both staff and management, are unwilling to commit. Except in fixing bugs and spending time on rework. Far too many people see that as an regular, accepted part of software development.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Thanks to everyone who attended my sessions today at SQL Saturday South Florida. Here are the resources from today.
Blogging for the Tech ProfessionalDeck: BloggingfortheTechProfessional.pptx
Get started with writing today. Use Word or Open Live Writer, save 10 posts offline, then set up a blog.
Architecting Zero Downtime DeploymentsI always enjoy this talk and I hope you did. Here are the things you might need:
- ArchitectZeroDowntime.pptx
- GitHub repo – https://github.com/way0utwest/ZeroDowntime
I got this in an email about a week ago from the PASS Data Community Summit.
There’s more to it, but essentially I submitted 3 talks (2 on deployments, 1 pro dev) and none were selected.
It’s slightly disappointing, as it is a rejection, but it’s not really a rejection of me. It’s that the talks didn’t make the cut. I’ve spoken there before, and I have other engagements coming up at events. I’m also still going to the Summit and I’ll likely still have some speaking slot from Redgate.
For others, this might be very disappointing, but it’s not a rejection of you as a speaker. It’s that there are only so many slots, and there is a lot of competition. The program committee is a volunteer one, and while the conference organizers review their selections, they don’t make many changes. I know because I was one of the final reviewers where we looked at the breakdown of sessions in different topic areas, the diversity, the experiences, and we really only made 3 changes out of a few hundred where we saw a gap.
I don’t want to give too many specifics, but Grant noticed we had no Extended Event sessions in there, so we removed another DB Administration talk and added an xEvent one.
This year had over 1,100 submissions for 108 slots. There are a few more slots for vendors and a small few for first time speakers specifically, but that is a lot of competition. My Pro Dev talk has been popular elsewhere, but it wasn’t better than the others that were selected. Same for my other submissions.
They were good, just not good enough.
I do get rejected by some events, as do most other speakers. I don’t know anyone that is picked 100% of the time.
It’s OK. It doesn’t stop me from submitting, and it doesn’t mean I wouldn’t do a great job at this conference. It just means not this time.
If you feel bad, that’s OK. Just keep submitting, working on your presentations, ask for feedback from others, and keep sharing your knowledge. You’ll get picked at another event.
One of the hot terms in software these days is observability. There are a few definitions (Splunk, RadixWeb), but essentially this is the insight into how your software runs and performs using metrics, logs, traces, etc. In DevOps, we do this with an eye toward improving performance and identifying the root cause of issues. The focus is slightly different from monitoring, where we often focus more on resources and health. We need both, but often in trying to improve software and the behavior for users, developers need observability. Infrastructure people responding to acute issues and looking to ensure we have the capacity, availability, and other x-bilities, that need monitoring.
Today I’m wondering if you collect a variety of types of metrics for your software that might tell you how your system is running. What things are important to you in order to better serve your clients? If you’re a DBA/sysadmin, what is important to you? If you are a developer, are there different types of data you want?
Certainly, you might collect various resource measures (CPU, IO, reads, etc.), but there are many more things. There are logs, which could include the SQL Server error log, but I’d hope that you had a more in-depth way of measuring the activity on your system. Do you have custom xEvent traces running? Can you collect application logs easily when you’re looking at issues? Do you spend time trying to solve chronic issues? Do you look for potential future problems?
Most software applications should include some sort of basic logging of major functions, but I would hope that there are various levels available. If problems are reported or noticed, can you increase the detail of logging? Can you correlate this with logs from different systems, such as the database? Can I get execution data that matches calls and trace down the potential issues that users are experiencing?
I know that we have problems in applications. Some (many?) of these are data-related, which might be easy or hard to trace down. That often depends if the data changes too quickly or is static enough for someone to investigate a report. Some errors are logical code errors, which might indicate a lack of testing early in the software process, but we ought to be able to determine this quickly from logs. If there are performance issues, and we have a lot of these, how easily can we verify a problem?
Let us know what types of metrics help you solve issues. I certainly think that you should have some sort of monitoring and observability system in place that helps you dive deep into the database, especially concerning execution plans at the time of the issue. The situation can change quickly inside a database, so capturing data regularly is important. If there is something you wish you had, let us know as well. Maybe someone else will have a neat solution for you.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
addleworth – adj. unable to settle the question of whether you’re doing okay in life; feeling torn between conflicting value systems and moveable goalposts, which makes you long for someone to come along and score your progress in discrete and measurable units – points, dollars, friends, followers, or a grade point average – which may not clear up where you’re going, but would at least reassure you that you’re one step closer to getting there.
This is certainly something I’ve felt, often with my financial future and career. I feel this when looking at my salary or my savings and wondering if I’m doing okay compared to others. Often, I’ve compared myself to others and tried to establish a comparative metric that helps me determine if I’m doing well.
This used to be dollars, and for the most part, I’ve given up on this. Once in awhile I look at someone and think “could I be like them” or “should I try to do what they do” to achieve something similar, but I quickly look at my self of happiness, stress, and other ephemeral metrics to realize I have a great life and shouldn’t bother measuring whatever I’m observing.
From the Dictionary of Obscure Sorrows
The last 10-15 years in software development have seen a widespread embracing of unit testing. Before the popularity of mobile phones and their apps, most of the organizations I’d worked in gave lip service to automated unit testing, and often even more complex integration/system tests.
These days, it seems more and more people embrace unit testing, and I hear about that often from customers and attendees at events. I don’t often hear about more comprehensive integration and system testing, I found an interesting article from the Pragmatic Engineer that looked at how the Bluesky social network was built. The article is partially paywalled, but I have subscribed because of the interesting thoughts they publish. In this article, there was a really interesting part of the article on testing. This is a section titled “Integration tests over unit tests”.
This isn’t described in any detail, but there is a note that the priority is for integration tests. There are unit tests, but I’m guessing this means that they write mostly integration tests first and then perhaps fill in with unit tests. The article does note that the backend is heavy on integration tests, looking to test the flow of data through the network. I assume network means their application here.
I think a lot of companies think about skipping unit testing in the database and instead might run some sort of integration tests from the application that hit the database. At least, I hope they do this. If they use unit tests that mock the database, that’s not necessarily a great way to ensure that the things in the database work as expected. Data types, defaults, rules/constraints, etc. All of these things might be different in a database over time, which is why devs need a database that is regularly updated from production (though is likely smaller in dataset size).
I can see some value in looking at integration tests more than unit tests. If we can only test so much because of time pressures, ensuring that the flow of read/write to and from the database makes sense. Queries as well, since potentially there are writes that store data and queries that read or aggregate data that might be transformed somehow in the database. Imagine writing a field, but having queries that read from a computed column that has normalized the data somehow. Or even triggers that have changed the data.
I am a bit fan of testing, and I think there is value in database unit testing, but I also understand that many people struggled to get started there. If you aren’t going to do database unit testing, then ensure that you are running integration tests that call through to a database. I realize this can be a hassle to set up in a pipeline but building better quality software means some investment in automated testing. That requires a way to build and update dev/test databases over time. A crucial part of embracing DevOps in the database.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
What topics do you want to see presented at a future SQL Saturday (or other event)? Steve Rezhener has built a survey that you can take.
Take the survey today!
You don’t have to answer every question. Pick those you are interested in and leave the rest blank.
I’ve set up redirects at SQL Saturday as well:
- https://sqlsaturday.com/survey
- https://sqlsaturday.com/surveyresult
I’ll do some analysis in a week or so of the results and publish something.
I recently had an issue in one of my Git repos, and decided to drop all my local changes and just pull down from the remote. This post looks at what I did.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. You can see all posts on Git as well.
A Bad StateThe old cartoon looks like this:
In my case, I hadn’t done this. I didn’t have a fire, but I did leave the building.
Actually, what I’d done was made a few changes at home and hadn’t committed them. I was in between trips and in a hurry, and walked away. On the road, I made similar changes and did commit/push them. When I got home, I couldn’t git pull because of the conflict.
What’s worse, these were binary (Excel) files.
I could have tried to sort things out, but in this case, I knew the remote copy was likely more up to date in place and I could easily re-enter the data I’d saved but not committed.
The way to do this for me, for tracked changes, was git reset.
In my case, I wasn’t trying to reset to a particular commit, I just wanted to whack all changes I’d made. This was just one file for me, so I issued:
git reset -–hard
The -–hard discards changes to any tracked files. Changes to untracked files aren’t affected. I’ll write about that in another post.
This cleaned my local repo back to the last time I’d had a git pull. From here, I could just get changes from the remote and work on.
SQL New BloggerThis post took about 5 minutes, literally, to write. Some of that is I’m a good typist, some is this is a simple story. Any tech pro ought to be able to do this in 5 minutes as well. If not, learn to type or to structure a short story.
This shows a little tech knowledge, but also an explanation of a situation.
There are lots of resources for learning: articles at SQL Server Central, blogs, user groups, SQL Saturday and other events, conferences, and more. In most of those cases, the editor, author, or speaker is deciding what they want to write about. If you want to learn something different, you need to go search out that information. You can certainly request topics from others, but they may or may not listen to you.
At least not as an individual.
Steve Rezhener put together a survey for what topics you’d like to learn about. A few others, including myself, gave him feedback and he’s published this for people to use. It was intended for SQL Saturday organizers and speakers, but it can work well for anyone producing information. I’ve created some shortlinks at SQL Saturday that you can use to take the survey and see the results.
- https://sqlsaturday.com/survey
- https://sqlsaturday.com/surveyresult
This is open-ended, and none of the items are required. It’s long, but just answer the items you care about. While it does ask for places you’d attend events, you can answer or leave this blank. I love surveys like this one, since I can pick and choose and don’t have to answer every question.
I plan on analyzing this data every month or so and publishing a report, which helps me decide what to request or publish here, but also which topics I might speak on in the future or what I might plan a SQL Saturday around. I would love to see more niche events, especially virtual ones. If some of you out there want to be an MVP, run a virtual event on your niche topic. Ping me and I’ll help you get going.
The results of a survey like this might also help you decide where you should drive your career. If a lot of people are interested in something, likely it’s relevant to their jobs. Perhaps you ought to follow the crowd a bit if you aren’t sure what things might bring opportunities for you in the future.
If you’d like to see more topics or different choices, drop Steve a note and I’m sure he can add to the form.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
The last few years at Redgate we’ve had the entire (or most) of the marketing department come to Cambridge in the UK for a week. A few weeks ago was my third time attending, and I once again had a good time and enjoyed the week.
While a lot of marketing works in an office, there are quite a few of us that are remote, and many people in different offices don’t regularly see or work with each other. This is a chance for us to cross normal boundaries and get together to bond, brainstorm, and better understand each other.
Not everyone came (I missed Louis Davidson ), but there were a lot of us. Our team picture is below.
A few years ago the week was packed with different activities. It was a bit much, and we’ve left more open time for business as usual (BAU) during the week so people can still have meetings with other departments or get things done. Some people do a bunch work, some socialize with others. For me, it’s a mix. While I get to Cambridge a few times a year, I never see enough people, so I do a mix of socializing and normal work, though I get relatively little done during the week.
MondayWe kick off the week with an opening brunch. In our new office, we don’t have a place to eat, so a few people bring in various fruits, pastries, etc. It’s a good time to chat with people in our large open space. Our CMO (Chief Marking Officer) reminded us in a short opening to meet new people and welcome the few newest people who had been here only a few months, so I chatted with someone I’d never met.
I also had a couple other meetings, and then headed out to a group social event. We met at a local park, carpooling over there, and split into some teams for a scavenger hunt.
One of the items was get a drink from a pub, so Grant and I did that.
I had to race back to the office after this for a weekly standing meeting, which is normally in the am for me, but this time it was at the end of a long day.
TuesdayI had to arrive early, as I had a meeting with a couple people from the Summit team. We brainstormed on a few things cooking for November, and it was a nice quiet way to start the day.
We opened the day with some RedTalks, which were short presentations from various people on the status of some of their initiatives. It was nice to see these in person, with many of them having 2-3 people sharing the presentations. These covered some marketing process things as well as how some of our initiatives were working.
As I listened, I browsed the snack table. We do this every year, asking people to bring snacks from their home area. Anyone can try them, and there is a donation request for a charity. As of late Wed, I was the only one who donated, so I hope that changed.
One of the interesting ones for me was on the takeaways from a Gartner conference on how marketing succeeds or fails in a company. A good reminder on coordinating with others, as well as the note that more doesn’t necessarily mean things are better. A good takeaway was tackling a new thing should mean we let some other thing go. I do this, but I know a lot of others don’t.
Lunch was again catered, and this time we were split into 5-6 groups in different rooms with one of our execs in our room. I didn’t get a chance to talk with our Chief Product Officer, though he and I had chatted a few weeks earlier, so I wasn’t too concerned.
Afterwards we had our expo, where each marketing team created a small board of things they do and challenges they face. We use some whiteboards and some paper pads. I have to say I was trying to get a couple things done and chat with others, so I didn’t do any prep here. Or any explanations.
Once the boards are done (we get an hour) the expo area is open to the whole company. Execs, other managers and some workers come by and check out what marketing does. I managed to get up there for a few minutes, but ended up chatting with a couple other managers and taking a call from my wife, so I missed most of this.
WednesdayI arrived early. I had an 8a meeting, but with the trains, I got there about 730a. I took a few minutes below here to sip coffee and play guitar. We have a couple in this atrium and I try to steal 5-10 minutes at least once a day and relax.
Another morning Summit meeting. For a crew that’s an hour behind me in Vancouver, they schedule early meetings in Cambridge. Here we reviewed the sessions selected (should be out now) to be sure we had the coverage we wanted in various areas.
We did suggest a couple changes were we were missing something and had some overlap in others. We left notes, though I don’t know as I write this if our suggestions were taken.
Then we had a recording sessions. Grant, Ryan, Louis, and I were asked to start a new podcast since the other Redgate ones had fallen away. There is a dedicated recording room in the office, so we spent a few hours on a couple episodes. No idea when they drop.
The afternoon was our annual social. We all made our way over to a local bar where we had a few activities and lunch. I was late as I had a meeting about my AI chatbot (more on that in another post), with a few action items for me given.
The social was pizza to start, then a quiz where we broke into teams and had a presentation where we were given 5-7 questions at a time in different areas. My team was semi-competitive, and I wasn’t competitive at all. Also, my back was a bit sore and I struggled to concentrate, but it was a lot of fun. It also helped that the bar wasn’t open until after the quiz to keep people focused. I learned a few interesting things, like red gate came from Italy and one of our founders built and released a video game. I know there were other things, but as I said, I was tired, hurting and unfocused.
I did enjoy sitting on the open rooftop and chatting with friends. Ones I’ve known, not new ones, but time is short and I try to balance time with new and current friends.
ThursdayThe Marketing Innovation Day, which I’ve had mixed feelings about. We’ve tried this in a few incarnations, but essentially we try to improve marketing at Redgate. This year we split into some groups, or were split into groups, and tackled the day in 5 sessions. The first was the creation of what an incredible year at Redgate would look like next year. This was from the perspective of a magazine cover, and it was fun to hear others’ ideas and thoughts on what success would look like.
Then we did sessions on:
- challenges and opportunities
- analysis of others’ ideas
- a plan to tackle a new thing
- presentations of the ideas
This was interesting. Each group was assigned either a challenge or an opportunity and we used lots of post-it notes to jot down ideas. We were on the positive, opportunity side, which I liked. Each session was about an hour, and the first one had us write down a dozen or so notes.
During the second, we walked around, trying to decide what we liked and what didn’t from others. That helped us decide on what we might tackle in the fourth session. Here was one I saw and one of my teammates liked.
The fourth session had us picking something. My group picked building an AI bot to help customers, and while I wasn’t sure about the idea, I warmed up a bit. We had to debate some of what we’d propose, the costs, upsides, downsides, etc. One of my teammates built the presentation, and they picked me to deliver it. That’s fine, but they wouldn’t show it to me, so I went in blind.
I had a nice crowd for the session, and I went last.
My group won the online voting 26-20-5-3 in the positive ideas. Both Grant and I delivered our group’s ideas. He got 20, so I guess that means I win in two ways?
Afterwards we had another meetup at a bar for a few drinks, which was nice. I got to chat with our CMO, which is always good. We never get much time together, so that was good. I also hung out with some US marketers who I rarely see.
FridayAnother early morning for me. Some coffee guitar, and another meeting. Then we had a final Summit brainstorm looking at things we might do for the welcome reception. I’m not sure we have consensus, but it did get me a bit excited for November.
We also had our final farewell at lunch with some early cocktails and goodbyes. Many of us won’t see each other until next June, so it’s both happy and sad.
I forewent the prosecco for a little lemonade and some of bottle I’d left behind pre-pandemic. Can you guess which is mine?
Years ago I realized I don’t like the regular Fri afternoon gin happy hour or the prosecco celebrations, so I bought my own bottle and left it there. This is the first time I’ve touched it since last year.
Closing Out the WeekIt was a semi-quite close out. After drinks, I had to do a few SSC things and then walked around saying goodbye to people. My wife had flown over from the US, and I left a bit early to see her and start our holiday.
Each year we’ve slightly changed how our Global Marketing Week works, and I think it is a neat experience. I don’t know if we are better at running the event, but it is a good chance to get our department together and think about how we do things and what we do. There are lots of random conversations about past and future, which don’t quite happen over Zoom.
For most of the companies I know are almost all remote, with people working at home most every day, they provide some opportunities to get together at least once a year. I don’t know who is 100% remote and never meets anyone, but I think something is lost if you never meet coworkers. That being said, once a year is nice for the entire department and I’m grateful Redgate does this.
In the last few months, I’ve been traveling around at a few of the Redgate Summits (one more in NYC coming) running panels on cloud journeys. I’ve had industry experts, both technical and managerial, discussing their approaches and journeys with advice and caveats for others. It can often be more than just migrating systems, so a lot of people have started to talk about cloud transformation.
However, in some cases, this is just a migration. A lot of companies just lift-and-shift their databases into the cloud, along with various other services. While this is a quick way to get into the cloud, it isn’t much of a transformation. If you review and right-size the resources you’ve provisioned, maybe there is a bit of a transformation, but not a lot.
Instead, the idea recommended by most vendors and consultants is to transform your software to work in the cloud. This might be moving to containers, to using more cloud-native services, or re-architecting your software to embrace to way cloud vendors provide services. Often this is a major project, though it can provide advantages over time with cost savings, more efficient code, and better-trained developers.
That last one is key, as a lot of the advantages of the cloud require your developers to write better code and re-think how they interact with data services. Code costs money, and poor code costs more money. This is also true on-premises, but it’s more true (and more visible) in the cloud.
I was surprised at how many companies had embraced the cloud and how many had seen savings. I met quite a few companies that had moved their databases (including large enterprises) 100% into the cloud. They’ve seen savings, but they also rigorously audit resources and the provisioned sizes.
The cloud isn’t for every organization, and not for all workloads. There are more than a few companies that have struggled to achieve the results they want in the cloud, be these performance or cost measures. I suspect some organizations will never move fully to the cloud, and may never move databases. However, it is a tool that we technical people ought to understand and learn why we recommend for or against a move.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
sayfish – n. a sincere emotion that seems to wither into mush as soon as you try to put it into words – like reeling in a shimmering beast from the deep only to watch it wriggle limply on the line, which makes you want to leave it down there, languishing unexpressed, where it’ll grow dark and slender and weird, with ghostly blue eyes and long translucent teeth.
That is an excellent description in a definition. A thing shimmering into mush as you look at it. I’m sure some movie has a great visual of this, but I think about something fading and dissolving as I focus on it.
I think I’ve sometimes had this emotion when I start to get annoyed or angry at someone (or a situation). As I start to process the emotion, it fades and disappears as I realize I’m being irrational.
Or more likely, I’m just hungry and over-reacting.
I don’t usually think it lingers or grows, but maybe I’m just repressing things. Who knows. Still a great definition.
From the Dictionary of Obscure Sorrows
I’m heading to SQL Saturday South Florida 2024 next week. This is my second time attending the event and if you’re in the Miami area (or want to take the train down from N Florida), register today and join me.
This is my second time attending the event after missing them for many years. Andy always said this was a fun place to go, but timing often didn’t work. It did last year and again this year.
I’ll be doing two sessions next week, but the schedule is packed with a wide variety of talks on ML, DevOps, Performance tuning, T-SQL, Fabric and more. I’m doing a talk on Blogging and one on Zero Downtime Deployments.
Register today and let’s connect, learn, and share next week.
I returned last Sat, 15 Jun, from 9 weeks of crazy travel. It wasn’t all for work, but it was a long stretch. I spent a good portion of Monday and Tuesday just getting caught up on what was going on, what the status of my workload is, and what I might have missed.
While it’s been a hectic week, it wasn’t that hard, other than a little recovery from jet lag. The last two weeks were in the UK/EU, so I don’t think I was back on Colorado time until Tuesday.
What helped me deal with that long stretch was some vacation. My wife and spent most of the second week in the UK in Tuscany, Italy. We rented a small place in the Cortona region, just outside of Camucia. We rented a car and visited a few wineries, as well as Siena and Montepulciano. You can see the edge of our farmhouse down a private driveway with Cortona on the hill in the distance.
We had a very relaxing time there, and I’d highly recommend it was a vacation spot.
Part of my time in Australia was also a holiday with my wife where we mostly relaxed and did little, but explored Sydney a bit. Waking up to this view was amazing.
The point of this isn’t to make you jealous or brag about my holiday, but to remind you that getting away is important and nice. It can help you recharge, even when you’re busy. My trip to NY to see my daughter graduate was busy and hectic, but even then I felt better coming back to work after the change in scenery.
Whether you want one long break a year or lots of small ones (my preference) make sure you get away from work.
SQL Injection has been a problem for my entire career. Thirty years ago I could have easily just blamed this on ignorance, as most of our developers didn’t think about the nefarious ways that hackers enter data in our applications. These days, there isn’t a good reason for this to keep happening, and the problem is us. I think that we don’t provide good examples or training on secure coding or secure architecture as a normal part of teaching programming. In many organizations, we don’t check for issues and prevent their release. Some do, but many don’t. On top of this, the existing code is usually a poor template for writing future code. I do think Microsoft aims for secure coding in SQL Server but in Windows, there is work to be done there.
A few months ago, I saw an article that noted the US CISA organization and the FBI issued a secure-by-design alert (PDF) that noted there is no excuse for SQL Injection vulnerabilities (SQLi) in modern software. This alert notes that SQLi has been an “unforgivable vulnerability” since at least 2007. Inside the document on vulnerabilities, it notes that a single quote can’t be used in certain fields: username, password, ID field, or numeric field. They also note that co-mingling user data and query data, like constructing queries on demand, is a poor practice.
The alert even emphasizes that developers are engaging in poor practices when they “fail to treat user-supplied content as potentially malicious.”
I agree, and their recommendations are worth reading and implementing. If your boss doesn’t want to spend time on these, point out the bulletin and note that since this is a published advisory, I wouldn’t be surprised to start seeing lawsuits in the US or even insurance claim denials if your software team doesn’t follow these practices. Note that the list includes leadership support of secure coding and secure design principles.
I doubt this has changed a lot, but I think some managers likely see this as a) a good idea, and b) a way to mitigate potential issues down the road. Changing the habits of software developers, updating code snippets or patterns, and even adding linting/static-code-analysis to CI pipelines take time, as does the training for developers. However, it’s something that has to start changing over time to get better at building higher quality, more secure software.
I’d like to see insurance companies refuse to indemnify or cover losses or problems from software that is written from this point forward and is vulnerable to SQL Injection. There are far too many tools out, and software is too critical to allow these types of simple coding errors to proliferate. I’d also be pressuring companies to ensure older code is being actively refactored to reduce the number of vulnerabilities over time for all software they still support.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I’ve seen the term polyglot persistence floating around Redgate a bit recently in the marketing department. I haven’t really seen this term anywhere, and I wonder if you have. If you have, drop a comment.
The definition is based on the polyglot programming, where you write an app using multiple programming languages. I don’t know a lot of people who do that, and if you do, let me know. I guess some do, since I’ll see things like HTML+C#, or maybe something like Blazor + C#.
In any case, the idea seems to be that you use multiple databases to satisfy your requirements. A few examples might be:
- SQL Server for most data, Redis to cache shopping carts
- Oracle for most data, ElasticSearch for full-text searching
- PostgreSQL for profiles and leaderboards, MongoDB for real-time game actions
I’ve tended to showcase NoSQL + Relational here, but I have seen a few people using two relational stores or two NoSQL stores because it suits their needs. The really common additions (for me) are some type of NoSQL store for performance with more stable, long term data in relational.
I don’t think this covers the relational for OLTP and something else for OLAP/warehousing, as that seems to be an ETL/ELT/transfer of data over time for a different purpose rather than for the same application, but I guess it could cover that.
I think Polyglot Persistence is a good idea, especially when you have complex requirements that aren’t easily solved with one database, even when you have in-memory or search/graph features in your database. Often those don’t perform as well as specialized systems, though there is a level of complexity and the challenge of ELT/ETL’ing the data to/from the second system.
Evaluate carefully, PoC, and test at scale before you decide this is worth the trade.
I almost called this “chasing a new laptop” since that’s what I’m doing, but I decided to add the date because the current laptop I’ve using was built in March 2019 and got to me in May 2019. I’ve had an HP Spectre x360, my second HP Spectre, and I’ve really enjoyed it. I’m also amazed it still runs. On the last few trips, the two rubber strips that run along the bottom (acting as feet) started to peel away. I’ve never seen that before and I tried to re-attach them a few times, but that didn’t work well.
Not a big deal, and I can live with that, but then during my Australian tour, the laptop started pausing and freezing a few times. It might be that there is too much software on there and needs a pave-and-reinstall, but I decided to check the age on the machine. That was when I realized it was five years old. It’s been a great machine, but I don’t think I’ve ever had a work laptop last that long with daily use. Of course, there was about a year during the pandemic when it was rarely used, so maybe its life lengthened during COVID.
In any case, I started looking around for a new one. My initial glance at smaller laptops has me considering these: another HP Spectre x360, the ASUS 14, the Lenovo Thinkpad X1 Carbon, and the DELL XPS 14. That’s the short list, since I really want a Windows PC that can run SQL Server and tools locally. I’ve had a Macbook before, and in the past I could run a VM with Windows tools. I can run SQL Server now, but not SSMS, and I like running SSMS.
I’m just starting to look at some reviews, and think about what I need. I’ve had a i7 CPU, 16GB RAM, and a 1TB SSD, which have served me well for the last 5 years. I don’t think I need more, though 32GB is tempting for a memory upgrade. I don’t do a lot of heavy development, so it’s more about having the things I run be snappy in presentations with customers and at conferences.
What machines do you use and like? I’ve appreciated the durability of the last two HPs, so I’m leaning towards another. I do like the pointer on the Lenovo, so that’s tempting. I’d like a touch screen as I use it regularly, though I don’t need a folding 2-in-1 machine. A normal clamshell is fine.
If you have a recommendation, let me know. If you have a dream machine, make me drool over it.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
One of the interesting things that I see at Redgate Software is how idealistic our developers and engineers can be. They often build our database DevOps products with the idea that customers will use well-designed databases. The systems will have primary keys, foreign keys, defaults, constraints, indexes, and more. Developers will use coding standards, and naming conventions, and will understand what data is stored in tables. Not in every case, but often.
After all, that’s how we build software at Redgate, as teams, sharing information, publishing documentation for others, and following best practices.
It’s cute and endearing, and unfortunately, not often true. In most cases, I find databases built by developers, accidental DBAs, or even experienced DBAs to be full of inconsistencies, lacking constraints and keys, and even duplicating some indexes and forgetting others. I often joke during one of my presentations that the main thing people should learn is to add primary keys to their tables. However, I’m not really joking.
During a recent design session on our masking technologies, there was a discussion on masking data in tables without PKs, which is a challenge. We’re working on it, and also on being able to mask PKs themselves, as some people use the PII data as a PK. This could be a tax ID of some sort, but could also be an email address.
When one of our account executives (Rob Boswell) heard that we were enhancing our capabilities with regards to PKs, he joked that we will soon be “primary key agnostic.” It was a great line, and in one sense it’s true. In another, it’s sad that we need to design tooling around such poor practices.
The reality of the world is there is a lot of bad design, bad architecture, and bad code out there. I applaud those who work to improve things in their environment, am saddened by those who don’t (either improve code or their skills), and frustrated by management not supporting efforts to be better. At the very least they should support efforts to teach your staff to code things right the first time, which helps improve future code. The next best thing is to refactor and improve older code, which can help you spend less in the cloud, or run longer with the resources you have on-premises.
The reality is the reality we are in, but that doesn’t always need to be our future reality. We can change the future, each of us, by learning to write better code and improve how we approach our work tomorrow.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
At the Redgate Summit in London, I ran a panel talking about Platform Engineering and how we can make developers more productive. One of the questions from our audience revolved around AI (Artificial Intelligence) technologies and how they might assist. As a note, AI tech includes a lot of different things, like machine learning (ML) among other things, but a lot of people seeing the media and hype around LLMs (large language models) think those are AI. They see AI as what is implemented in ChatGPT and Copilot, which is correct, but incomplete.
One of the panelists, Jeff Smith of Redgate, said that he views the output from AI as a first draft, something that bootstraps further work by a human. This can save time and can help someone be more productive, but it’s a starting point and a boost, not a final product.
I had not thought of AI this way, even though I expect to have to test, refine, and edit the results I’ve gotten from various assistants. While it might be possible for me to keep refining my prompt and perhaps generate better results (code and/or text), I don’t know that it’s often worth the time to do much of this. I might do a little, but it’s much easier to take the result of AI as a first draft and then make it fit my requirements.
In much the same way I use Stack Overflow or SQL Server Central for code snippets. They often give me a starting point and something I can then adapt to my needs without all of the time and effort spent on reading an API or SDK, experimenting with the syntax, and then actually using this in some code I’m writing. Similarly, I can get a head start on writing a summary or a pitch, though I’ll admit I’ve found it less useful there. In fact, for many of my editorials, I’ve tried to use it to summarize an article or web link, and it does a very poor job (IMHO).
Some of you might be concerned or worried about this, and some might not. To me, the average-to-poor code produced isn’t any different than the legions of Stack Overflow developers or SQL Server Central DBAs who take code from an online forum and fail to test it. Anyone using any code (or prose) should test (proof) it and verify it works appropriately in their situation. We have a lot of people who don’t do that now and AI might exacerbate that with more code, but I don’t know if we are in a worse situation. We certainly seem to be able to produce lots of average tech professionals now who turn out plenty of average code.
Ultimately AI is a tool that does exist in the world and will continue to exist. We have to learn to work with it and use it to our advantage. Good developers and DBAs will do that. I’d like to think that many organizations will look to hire and use those people as their staff and use fewer of the people who don’t bother to test and improve the results from their tools.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I saw a tweet that DBCC CLONEDATABASE was being discontinued for production databases, which both scared me and didn’t make sense. I’ve used this a few times for a quick copy of a database and like how it works. Discontinuing it seemed strange to me.
Then I read the blog post, which notes that it’s not being supported for production deployments. The post doesn’t explain why, but I’m guessing this is because all the stats and other metadata moves, and this might mess up the optimizer if different data is added. I don’t know who deploys production databases like this, but I could see people who have federated or sharded databases using this to create a new blank copy and then uploading data into it. Or maybe people who need new databases that are distributed onto remote office/edge devices used it? If you use this to create production dbs, let me know.
This command will still be used for generating schema-copies for diagnostic and troubleshooting purposes, which is what it’s there for. I assume this means Microsoft Support will support you using this to investigate strange query issues or if you create a database at their request. However, there isn’t a mention of this being used for development and test environments, which is where I use it.
Specifically, I’ve used it quite a bit lately with Redgate’s subsetter utility. I need a target that’s shaped like the source, and dbcc clonedatabase gives me that. I don’t really care about stats or anything else, I just need schema set up to move data around. It’s useful there. It’s also useful for a quick test of a deployment, where I can ensure I get the latest production schema and then run a deployment against it (hopefully using Flyway), looking for errors.
If you use this command, don’t worry. The tool isn’t going away, and the restriction against production copies doesn’t take place until March 1, 2025. That’s nearly a year from now to change your process. If you’ve never used it, well, I don’t have a good reason to start using it, but you ought to be aware of how it works. Learning about new features is handy, as you might discover a problem the knowledge can solve. However, if you don’t know it exists, it will never enter your mind as a solution.
Be curious and try things. It’s easy, and it’s fun.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
I’m in Cambridge, UK today, at Redgate Software. This is our annual Global Marketing Week, where the entire marketing department gets together to learn, share, connect (recognize the theme), as well as enjoy some time with colleagues that we rarely see. I think Grant, Ryan, and I see more people more often than most others, and I’m grateful we get to spend a week here.
This is the end of my wild travel time, where I’ve spent 11 full days at home since Apr 12. In fact, I don’t get home until Jun 15, so it’s not over yet, but it’s close. I’m definitely tired and a little worn out, but I’m looking forward to a week in the office and then a week of vacation next week in Europe with my wife.
Not much blogging for the next two weeks, though I may drop some office or fun pix here later this week.
Then I get home and no travel for a few weeks.
I delivered a talk today on database deployment best practices at the Denver Dev Days 2024. This is a great event, and I have been lucky enough to attend a few and speak.
The deck I presented from is here: BestPractices Database Deployments_DenverDevDay.pptx
The main sections covered in this talk:
- Feature Flagging
- Rollback/Undo
- Release Branches (Cherry Picking)
- Observability
- Best Practices
I’ll do some blogging on these topics separately in the future.
I have worked for a few startup companies, including SQL Server Central. Each has been a different experience, and I learned a lot at each stop. However, I’m not sure I’d want to go through that process again at my age. I was thinking about the challenges and the excitement of being at a startup while reading about the founding of Reddit. The post doesn’t go a lot into the technical details or the working life, but it is an interesting read from a VC investor.
I also found this post on Choosing Startup Life, which talks about what the author thinks about before trying to start a company. He compares this with life in a Big Tech company, which relates to lots of companies, in technology or not. The main differences are lower salaries, less infrastructure, lots of work, and upside in a startup. Big companies have higher salaries and more perks, less stress and responsibility, and not a lot of context-switching. In general, that’s been true in my experience, though in bigger companies that didn’t think they were software companies, I sometimes could end up with a lot of context-switching.
When you work in a startup, I hope you have some passion or belief in what the company is building. I think that’s what drives a lot of Kickstarter/Indiegogo projects. Someone wants to start a business to do x and they believe in x. It’s important to do that since you are likely to need to work extra hours. Startups often are cash-strapped, so they don’t have a lot of services and infrastructure and depend on employees to work above and beyond their main job. They also are in a hurry to get cash, so they want to complete their work sooner, which means more hours per week. Not all are like this, and once there is some investment capital, hours can ease a bit, but investors want their return, so this isn’t a slow-paced, relaxed atmosphere.
There isn’t a lot of security. I’ve had startups fail under me where we didn’t have enough money to make payroll. I’ve had them sold in an instant with the new owner deciding to drastically change the employment status for many people. At the same time, I’ve had some jobs that result in unexpected windfalls of money. I haven’t had a company go public yet, but perhaps that will happen at some point.
I think about being in a startup like being in my first apartment in college. Nothing is provided, I don’t even know what I’m missing until I look for it, I keep long hours there, and everyone is responsible for everything. There are also a lot of arguments over who should/needs to/ought to do what.
Is it worth it? I think that the chances of a startup becoming a success can vary. It is important to understand valuations if you are trading away hours of your life on a gamble you’ll make money. Keep an eye on the number of shares offered and the potential value they would have to reach to make a difference in your life. For that matter, make sure you know what a pile of money means for your life. USD$100,000 sounds like a lot if you got a check tomorrow, but how much would that change your life?
If you decide to create a company and take outside investment, make sure you really understand finances.
Startups are exciting, but make sure you like the people working around you. You’ll spend a lot of time with them, and if you don’t enjoy their company, you won’t enjoy your job. Startup life isn’t for everyone, but it can be an exciting chapter in your life.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
desanté – n. the brooding delirium of being sick, which makes time slow to a trickly and turns even the most pathetic of tasks into monumental struggles, until the act of lifting your head from the pillow feels like trying to climb a mountain, wondering if you’ll every find your way back again, or even catch your breath.
I had desanté when I caught COVID the first time. I struggled with energy, feeling incredibly listless and tired. At times I was fine, but at others, I collapsed into bed, not even wanting to lift my head to watch TV.
I also felt like this when I had the flu in 2005 or 2006. A miserable couple of days where I didn’t want to do anything by lay in bed, not read, not watch anything, just lay, suffer, and attempt to sleep.
However, most of the time I just keep going through colds, through strep, through various other illnesses. I struggle to sit still.
From the Dictionary of Obscure Sorrows
When we build software, many of us use the same algorithms to solve problems. We might choose a similar method for a quicksort or a lambda validation or a regular expression. For database work, your code for a running total (or other common challenge) is likely very similar to many other people. At least on the same platform. You might solve this differently in SQL Server and Oracle, but for the same type of database, many of us write very similar code.
Actually, many developers might copy and paste an answer from SQL Server Central, Stack Overflow, or another site. I’m not sure if I think this is good or bad, as it’s a good idea to reuse code if it solves the same problem. If you copy it and don’t test it, that’s bad. After all, the code might not solve your slightly different problem if you don’t check it.
In the modern world, if we build software for our business using an AI assistant, could our company be liable if we knew our competitors were using the same AI service? Is this any different than a human developer performing a copy/paste from SQL Server Central? I don’t think it is in many cases, though the same concerns about intellectual property might be present in either case.
The concerns over AI seem murky in some sense, especially as the AI might “generate” code that isn’t directly available on some public resource. I do think that this is more of a collusion using a service than an algorithm. Still, in the hyper-connected world, where many of our applications might look to take advantage of some service instead of implementing it ourselves, this could be an issue.
I ran across a piece that discusses a lawsuit about a common pricing algorithm being used by different hotels. In this case, it’s not that the developers at different hotels used the same code, but rather that the hotels used the same service from a company, which of course, used the same code for all their customers. Whether you think this is a valid lawsuit or not, this is the type of legal action that others might bring if two competitors ended up using the same AI service and developed very similar code that might behave the same way.
I don’t think that AIs (at present) can actually develop new algorithms or solve problems in a new way. Instead, they predict the likely solution based on how they’ve been trained by similar scenarios. In that case, how concerned are we about how getting common solutions in disparate pieces of software? For most of us, I think we’ve be pleased that we have well-tested (hopefully) code that runs efficiently (again, hopefully) being re-used in many places. That would be better for most systems in the world.
What isn’t better is if humans become more adept at specifying prompts and producing software without lots of specialized expertise. For many developers that might be average, or even slightly below average, this might cause them concern for the security of their position. With good reason, as labor is one of the most expensive parts of building software.
Ultimately, just as with any other position, the best way to build a safe, secure career, is to continue to build your skills and produce value for your employer. That way it’s unlikely any AI will ever outperform you.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Recently I was trying to delete a folder and kept getting the “something is using this, try again” dialog. It was annoying, but I couldn’t figure out what process had this folder locked. I know there are ways to do this, but I decided to ask Copilot how to do this.
I did this in the Edge browser, since that’s the easy thing. Note: I was actually on Windows 10, not 11.
I got this response.
I then hit Ctrl+Shift+Esc and got the Task Manager. From there, I can see Resource Monitor at the bottom of the Performance tab.
I opened this and it started in the CPU tab. I typed in the name of my folder, and I saw that there was a CMD window open with the handle. I then realized I had a window in my list, which I exited.
Once I did this, I could delete the folder.
It This Better?Not necessarily. If I run a search, I see lots of results. In this case, any of these could be clicked, but in the past, I’ve often had to click multiple results to find something that works. I supposed Copilot could have given me a poor answer, and I’d have to refine the prompt, so the process might be the same.
Except.
Here I am working in one place and not clicking other links, potentially finding down sites, 404s, or other issues. It feels slightly cleaner. It also means I can have context if I refine my search.
I don’t know if it’s better, but I find myself leaving an Edge tab open to Copilot to try stuff.
Several years ago, I heard about a new product coming in Azure that would provide an IaaS (infrastructure as a service) VM to run SQL Server, with Microsoft managing most of the admin tasks for the instance, like patching and backups. That didn’t seem like a big load to me, and I wondered if anyone would actually pay for this product. After all, don’t most companies find managing patches and backups fairly easy to manage?
That product became Azure SQL Managed Instance, and I’ve been surprised at the adoption. Quite a few clients have adopted this as a way to lift and shift (mostly) to the cloud in an easy fashion without the restrictions of Azure SQL Database. This looks like a “normal” on-premises SQL Server, and there are both high-performance (Business Critical tier) and average-performance (General Purpose tier) versions of the product that let you choose what level of price/performance you need to achieve.
I’m curious today, and I have a question. What are your impressions of Managed Instance (MI)? Whether you use it, you have heard of it, or if you just read this description. Give me a few thoughts on whether this makes sense, performs well, or has issues you need (or wish would be) addressed.
I’ve heard there are some issues with I/O, but I also have clients who find it performs very well for them. I hear similar things from on-premises SQL Server instances, so I often think that either the software is designed well or the hardware architecture doesn’t match the workload. There have been a lot of enhancements to MI since its release, including the ability to backup and restore to/from SQL Server 2022.
There is even an offer from Microsoft that lets you try out MI for free (for a period of time). This is a way for you to test migrate a database to the cloud and measure the performance. You might need to do some work to measure your current performance in a way that lets you determine how MI stacks up. You will also need to do some financial number crunching to decide whether there is an ROI that makes sense. If you do that, be sure you reach out to your internal finance people to understand the differences between CapEx and OpEx expenditures for your analysis. Paying $50k a year for an MI license isn’t the same as spending $50k for a server and hosting.
I’m not sure what I think of MI. Like many offerings, I think there are places where it makes sense and places where it doesn’t. It’s not a simple decision for me as an abstract question. For specific situations, I might lean one way or the other, but I’d want to do some workload analysis to justify or discard my initial thoughts.
Share your thoughts and impressions today. You might help some of us learn more about why we might or might not use MI. You might even help clarify your own thoughts by writing them down.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Most of the people I know who speak at a SQL Saturday or user group aren’t paid for their efforts. At many of the community events, the speakers are volunteering their time. Many are also paying for their own way to those events not located in their area. A few, such as me, might get a company to cover their travel expenses, but often this doesn’t include time. If I attend a SQL Saturday, I still have a bunch of work on M-F that needs to be done. No comp time for these events. That being said, I’m happy to donate some time and money to community events.
Some speakers build and teach full-day sessions, usually called pre-conference sessions, for which they are paid. There can be competition at large events like the PASS Data Community Summit and SQL Bits to get a session since the payment can be rewarding. I’ve seen some speakers make USD$1k or so, which can cover travel expenses, and others make over USD$10k, which is a nice payday.
Most of us don’t want to teach a full day of sessions, usually because it’s a lot of work to build a day of training. It’s also very nerve-wracking to try and teach people who have paid you. A lot of speakers don’t really want to deal with that stress. I’ve done it, and it is hard work. I don’t deliver pre-cons at events because I get expenses covered at lots of events and prefer not to compete with others who might want to earn some money while growing their careers.
However, lots of people want to share their knowledge and teach others something useful. It’s been amazing to me how many people have stepped up to submit sessions and present them at events all over the world. I’ve been lucky and honored to meet many of these volunteers and call them friends.
When Andy, Brian, and I started SQL Saturday, we weren’t sure there would be enough speakers to run 10 events in a year in 10 cities. At the time, the PASS Summit and a few other conferences were the only places to speak outside of user groups and we weren’t sure there would actually be enough people in a community willing to speak to run a conference. We also weren’t sure that many people would travel outside their home area to speak.
I’m thrilled we were wrong. So many people have volunteered their time and energy to build a session and then deliver it at user groups and local events that there are often more speakers than spaces available for them. I’m also glad that so many of you attend and support local events.
Speaking can be intimidating, but I know many of you can do it. It’s scary, and it’s something I never thought I’d do when I left university. It is a lot of work, it interrupts your free time, and it doesn’t pay you any hard currency after a session. However, it’s also thrilling to help others, exciting to have them listen to you, it hones your communication skills, and it is impressive to employers. On balance, I’d say that even delivering a presentation at your local user group is a profitable endeavor and a memorable experience.
I hope we continue to see more of you willing to deliver a session for your peers, share some of your knowledge. I also hope that we see more of you stepping up to help organize a SQL Saturday or another local event in your community.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Recently the Flyway Desktop (FWD) team release support for state-based deployments. These are similar to SQL Compare deployments, but with your code source control, which is where you want it. This post looks at how this works.
I’ve been working with Flyway Desktop for work more and more as we transition from older SSMS plugins to the standalone tool. This series looks at some tips I’ve gotten along the way.
State-based ProjectsWhen I create a new project, I now have a choice: migrations or schema model (state-based). You can see the choice below.
I’ll demonstrate this now by creating a new project. I give it a name, folder, type, and then I pick the Schema model deployment source.
Once that is complete, the project is set up and I need to link a development database. I’ll do that first, picking a database with a few objects. In this case, I’ve picked my FWState_1_Dev database.
Once this is linked, the FWD tool will read the database, compare to my filesystem and tell me which objects have changed. Since this is a new project, all the objects are new and show up. I’ll select them all and click “Save to project”.
This gets all the code for my objects into the file system (and a VCS). I won’t commit these now in this demo, but normally I would save this code.
The main change in FWD is the change on the left side menu. Where we have “Generate migration scripts” and “Migration scripts” as options for a migration based project, for a state-based project, we have just a “Deploy” option. I’ll select that.
Once this happens, I have to pick a target. I haven’t configured anything (this is a new project), so I’ll select the “Manage environments” in the upper right section of FWD.
When I do this, I see the environments I’ve configured. So far, I’ve only configured a development environment, but I can click “Configure new database” to add a downstream target.
I get the standard FWD connection dialog, and I’ll enter details for my QA database. I usually also click “Test” to ensure I haven’t typo’d something. You can see I didn’t make a mistake below.
Once this is added, it is selected by default. However, I can always click the radio button in the Manage environments dialog and click “Confirm”.
This brings me back to the FWD screen where I see the changes that exist in the project, but haven’t been deployed to the target. In this case, it’s everything (QA is empty). I’ll just click the tbl_Customers table.
When I click Deploy, the script is generated just as it would be in SQL Compare to update the target with the code from the source. I can review the script, as well as change options. I can add an explicit transaction around all changes, or let them run with the default implicit transaction for each statement. I can also copy the script to the clipboard if I want.
If I check QA, I see no tables.
I’ll click “Deploy now” in FWD and the deployment starts. I confirm this is what I want to do.
Once this is done, FWD returns to the screen of objects. Note that tbl_Customers is no longer listed. The project and the target are in sync for this object.
If I go back and refresh QA in SSMS, I see the table exists.
SummaryThis short post shows how FWD and the Flyway system can be used to perform state based deployments, similar to SQL Compare or SQL Source Control. If you are used to working in SQL Source Control, this is an easy transition for you to a more modern tech, which will also support Oracle, PostgreSQL, and MySQL.
Try Flyway Enterprise out today. If you haven’t worked with Flyway Desktop, download it today. There is a free version that organizes migrations and paid versions with many more features.
If you use Flyway Community, download Flyway Desktop and get a GUI for your migration scripts.
Video WalkthroughI made a quick video showing this as well. You can watch it below, or check out all the Flyway videos I’ve added:
insoucism – n. the inability to decide how much sympathy your situation really deserves, knowing that so many people have it far worse and others far better, that some people would need years of therapy to overcome what you have, while others would barely think to mention it in their diary that day.
I really struggle to think I deserve sympathy for the challenges I might face in life. I know that I am very lucky and blessed in life, without many of the struggles that people who have less than I experience. I know that most of the time I have #firstworldproblems, which aren’t really problems, but annoyances.
I needed ankle surgery a few years ago, but really, it wasn’t critical. I could walk, I could bike, I could do yoga inside and snowboard in the winter. I was in some pain, but it wasn’t really a problem that deserves much sympathy.
I ate and drank way tooooooo much during the pandemic. Changing those habits was a challenge, but it wasn’t something that I think needs sympathy.
I have some tough travel schedules at times. I’m returning from Kansas City this week, and in the last 5 weeks, I’ve been home for 6 nights. Some of that is vacation with my wife, but a lot of work and travel. I’ve covered easily 30,000 miles in that time. However, it’s not all a burden, and it’s my choice.
Some people might think that’s an easy life. Some might think this is overwhelming. I think on balance, I don’t have a lot of insourcism because I know that most of my situations don’t deserve sympathy.
From the Dictionary of Obscure Sorrows
There is a nice article at Harness.io on their use of feature flags and how they deployed their next generation experience. It’s worth a read if you want to improve your database deployment experience, especially if you want to control how and when you release to customers.
A nice bonus is that they mention Flyway as a tool to help them manage database code changes. Hint from me is that you can use Flyway to not only help manage DDL changes but also DML changes as well.
I talk about this in my Architecting Zero Downtime Database Deployments talk. Most of the time when you have changes that might disrupt users, take a lot of time, or require coordination with different apps/systems, using feature flags and backwards compatible deployments is what makes zero downtime, or minimal downtime, possible. If your changes can’t be backwards compatible, you break the database changes into multiple steps that are backwards compatible.
Flyway makes it easy to ensure scripts run once, they are deployed as a group or individual transactions, and they run in order, so you can test your scripts in multiple environments.
I know I sound a little sales-y there, but Flyway is a fantastic tool and I was pro-purchase when Redgate bought the tool. I was also pushing to replace some other internally-developer deployment tech with Flyway in all our products, which we’ve done.
The big way Flyway supports feature flags is by ensuring you can break up your changes into separate scripts and limit when they run. This is best managed with branches in your VCS and PRs, but this also will be handled by the new Deployment Rules, which are in preview.
This article has a concept I’ve never heard about: invisible downtime. This is the idea that there are problems in your application that the customer sees. Your servers are running, but the application doesn’t work correctly or is pausing with a delay that impacts customers. From an IT perspective, the SLA is being met and there aren’t any problems. From a customer viewpoint, they’re ready to start looking at a competitor’s offering.
Lots of developers and operations people know there are issues in our systems. We know networks go down or connectivity to some service is delayed. We also know the database gets slow, or at least, slower than we’d like. We know there are poor-performing code and under-sized hardware, running with storage that doesn’t produce as many IOPs as our workload demands. We would also like time to fix these issues, but often we aren’t given any resources.
The current buzzword among executives and senior IT leaders is observability. It’s the goal of looking at how our entire system, application, database, and network, are linked and performing with an eye on improving performance. Not because they want to spend time or money here, but because customers are becoming more fickle and quick to move to another offering. Leaders know that degraded application performance (another phrase for invisible downtime) can have real bottom-line impacts on revenue.
There are a lot of products in this space, application performance monitoring (APM), designed to look at lines of code and determine how well each is performing. They can help you spot issues in application code, but they lack insight into database and network details, at least at a level that the experts need. As a result, digging into performance issues and root cause analysis of problems usually means pulling data from multiple sources and correlating log entries.
This is likely an area where AI/ML technologies can help, especially across large estates, though I think in many cases, what we need is just a pointer to poor-performing code. C#, Java, SQL, whatever. We need to know where the bad code is and then we need to train developers to write more efficient code. That might be the best way to improve application and database performance.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Recently I’ve been looking at archiving some data at SQL Saturday, possibly querying it, and perhaps building a data warehouse of sorts. The modern view of data warehousing seems to be built on using a Lakehouse architecture where data moves through different phases, but much of the data is stored in text files, often parquet files.
As a start to this I decided to try and move data to parquet. This post looks at writing parquet files.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
Writing Parquet FilesIn a previous post I looked at reading in JSON data, which is how some of my data is archived. I also talked about importing modules. There is a module, called pyarrow, that allows me to work with various parts of Apache Arrow.
One of the submodules in pyarrow is the parquet module, which lets me read and write parquet files. So, let’s get those modules.
import pyarrow as paimport pyarrow.parquet as pq
I am giving these show names so I can refer to them in code. Now, let’s skip the code from the previous article and assume I’ve got a dataframe with my sessions in it. How do I get a parquet file?
Fortunately, I don’t need to know anything about the physical structure, as I can use the write_table() function from the parquet module to do that. I’ll also use the pyarrow.Table.from_pandas() function to get data from the dataframe into this module. This code does that (with some setup for a filename).
outputFilename = f + '.parquet' outputFile = join(outPath, outputFilename) pqtable = pa.Table.from\_pandas(df)# Write Arrow Table to Parquet file pq.write\_table(pqtable, outputFile)
Note: I don’t know the technical differences between how pandas dataframes and the pyarrow tables work. I found a few notes online and it looks like pyarrow tables can handle more complex data structures.
Once this code is added to the code from the previous article (it’s already indented), this will write .parquet files to the bronze folder underneath the location from where it is run. In essence, this takes data from the raw folder and writes it to bronze in a new format.
SummaryThis post shows how to write parquet files out from JSON data. Take the previous article and this one and you can move data from JSON to parquet.
This code isn’t perfect. In fact, it needs work. I am only moving session data, so only a portion of the JSON data. This code should be enhanced, or the file names changed to reflect that, but for now, this is a quick example of producing parquet data.
SQL New BloggerThis post took about 10 minutes to write once I had the code working. In fact, adding these functions to the code from the last article only took a few minutes. I had to debug a few things to get the files into the correct folder, but it took longer to get these words down than get code working.
Not a lot longer, but longer.
You can do this. If you want to work in modern technologies, learn them. Learn how to work with parquet, which is being used a lot in data warehousing, and then write about it. Prove you can get things done and your current employer, or your next one, might give you a project to actually do this work.
Kubernetes is cool, and I think it’s really useful in helping us scale and manage multiple systems easily in a fault-tolerant way. Actually, I don’t think Kubernetes per se is important itself; more it seems that the idea of some orchestration engine to manage containers and systems is what really matters. As a side note, there are other orchestrators such as Mesos, OpenShift, and Nomad.
However, do we need to know Kubernetes to use it for databases? This is a data platform newsletter, and most of us work with databases in some way. I do see more databases moving to the cloud, and a few moving to containers. I was thinking about this when I saw a Simple Talk article on Kubernetes for Complete Beginners. It’s a basic article that looks at what the platform consists of, how it works, and how to set up a mini Kubernetes platform on your system. It’s well written and interesting, but …
Do we need to know anything about it? Are we running databases in containers, or will we? I think it’s possible that we might run any of our databases in containers. They are like lightweight VMs and there isn’t a reason why we wouldn’t run a database in a container. With external storage, of course, which gives you a cluster-like environment where your storage moves to a new node if the first one fails. That’s a good use case. Deploying consistent environments quickly is a good use case. Using Kubernetes to manage the containers is great, but …
I don’t think we need to know much about Kubernetes. I don’t think most of us should run it and should outsource any container orchestration to the cloud if we decide to implement database containers. These orchestration engines are quite complex today, and there is a lot of expertise needed to manage them. I don’t know that expertise is worth trying to find, train, and retain for most organizations. We should just outsource the container management to someone else.
We might need to know how we change the configuration of some resources, but that’s minor knowledge, and really, I suspect that outsourced K8S (shorthand for Kubernetes) will have GUI tools that let you easily pick and choose the CPUs, memory, etc. and then an export of the JSON or YAML or whatever is needed for the config. Most of us likely need the skills to export, save (in a VCS) the files, and then submit them to the cluster.
A few years ago I went through a bunch of courses and reading material on Kubernetes. I set up some small clusters, I experimented with pods, I even was excited to think about managing containers for various services. What I discovered is that Kubernetes is complex, hard, and something I want someone else to run. Once I set up a cluster in Azure I thought I’d never want to do this on-premises again.
Much like email. I have run email servers, but …
I’d like to never run one again, which is how I feel about Kubernetes.
I think containers have proven more complex and harder to work with than many people thought. I know there are plenty of people using them, but it’s a minority. I see many more organizations still building monoliths, or microservices that run as processes, or client-server apps. Not that many people are excited and using containers. That may change, and if you go to the cloud, containers give you portability that many other solutions don’t, so I’d recommend them there. However, they are still a bit immature, and hard to manage. I think it will be a while before we see lots of databases on containers.
Even if we do, I’m not sure we need to learn Kubernetes as database people.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
flichtish – adj. nervously aware how much of your self-image is based on untested assumptions about yourself – only ever guessing how you’d react to a violent thread, a sudden windfall, a huge responsibility, or being told to do something you knew was wrong..
I used to think I had all the answers and knew how I’d react to situations or what type of person I was.
I was a teenager. I think this view is fairly common among teenagers, but also in older people. I see too many people with confidence in how they’d react to an unknown.
And I see too often they don’t actually react that way. The more self-aware people recognize that their self-image was untested and wrong. The less self-aware people think the situation was an anomaly.
I know that until I walk a mile in my shoes in some situation, I’m very un-tested. I have a guess as to what my self-image is, but I doubt it is 100% true. As Mike Tyson says, everyone has a plan until they get punched in the mouth. Flichtish is a recognition of that saying.
From the Dictionary of Obscure Sorrows
We’re taking the roadshow across the water. Hope the plane makes it.
The Redgate DevOps Roadshow comes to Ireland and the Northern UK in June. We’ll be in these cities on these dates:
- June 11 – Dublin (Grant)
- June 13 – Glasgow (Grant)
- June 14 – Manchester (Steve)
We’ll also have one of Redgate’s amazing Solution Engineers, Huxley Kendall. He’ll be there to answer in-depth questions on the products and solutions that Redgate offers.
This is a great chance to get exposure to the solutions Redgate offers for building and managing your database code, as well as ask questions about the challenges you face and how we might tackle them.
Register today and join us for a great day in one of these cities.
I’m halfway through my crazy stretch of travel. I’m leaving today for Syracuse to see my daughter graduate college on Saturday. This is a nice trip, and a little break, but it’s still time away from home.
I’ve been home 6 full days since Apr 14, with trips to the UK and Australia in that time. I have work trips next week, the following one, and then two weeks in the UK again. All told, in 9 weeks, I’m at home for 11 full days.
My year so far:
I’m coping so far, though I am a little worn out with the movement through time zones. I’m going to use the next few days to take it easy, relax, and try to just enjoy the time with family away from home.
I’m off today, in New York helping my daughter get ready for her college graduation and likely trying to pack up an apartment.
While I lightly work, you get to re-read Learning from Exercise.
Hopefully I find some time today to get some.
This month is a great topic to me. I think growing and improving your career is a skill that most of us could improve, especially in our younger years. The invitation from Kevin Feasel is a good one from which you can learn a lot.
I am looking forward to the responses from others.
If you want to host an invite one month, ping me and request a date. Most of 2024 is full, but I have a few months, and I certainly am happy to schedule you into 2025. This is a great way to participate in the community, meet others, and challenge yourself. You just need a blog.
In this post I’m going to give two questions, one as an interviewer and one as interviewee.
My Favorite Interview Question for CandidatesWhen I interview someone, I usually have a list of things to ask them to better help me compare candidates, but these are associated with digging into knowledge, however this question really helps me.
What have you learned recently?
I don’t expect candidates to know everything. I expect to have to teach them quite a bit about my environment. However, what I want from them is an effort to learn. My view is that some people are constantly learning things and others are content to rest on their previous knowledge/experience.
If someone hasn’t learned anything recently, I don’t necessarily write them off, but I might probe about what they have been doing, as well as how they prepared for a new job or the interview. Perhaps they’ve been busy with something (crisis, illness, etc.) and haven’t been improving in the short term, but if someone hasn’t learned anything in the last year they’re proud of, or they can’t remember when they last invested in themselves, I have a hard time investing in them as an employee.
Note, I will dig into ensure you learned something and aren’t just giving me an answer.
My Favorite Question as an IntervieweeIn a lot of my jobs as a technologist, or a data professional, the job is the job. It’s very similar in many places. These days I do more architecture and advocacy, but if I were approaching a new job, I’d ask this:
What are the expectations around working hours?
I’d add context to this, but what I’m looking for are information for these items:
- core working hours
- on-call/non-core hours
- punctuality
I don’t mind working hard, but I don’t expect to work a lot of non-core hours every week, or even too regularly. I don’t mind 40 or even 50, but beyond that I’m not going to be happy.
I’m also not someone that punches a clock. If you expect me to be online (or in an office) every day at 8am, you’re going to be disappointed. I might be there at 7:45 or 8:15. I don’t avoid work, and I do my best to be early for meetings, but if nothing is scheduled, I will vary my start time. I usually warn a potential boss about this.
I used to ask about travel, but I’m over that. I don’t mind or worry about travel too much.
Recently I’ve been looking at archiving some data at SQL Saturday. As a start, I needed to read some of the archive data I have in Python. This post looks at the basics of reading in JSON data in Python, one of the more versatile languages for working with data.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
Reading JSON Files in PythonI have some data in the SQL Saturday repo in JSON format. This is schedule information, which is exported from Sessionize. I also have XML data, but I decided not to mess with that for now.
Getting this data into a dataset is actually easy in Python. Here are the basics. First, we need to import a few modules. In Python, lots of functionality is from various modules, which aren’t available until added to your workspace. However, they are easy to import.
We need a few modules:
- json – used to work with json data
- os – used to work with files and call OS functions.
- pandas – used for creating dataframes
- chardet – functions to detect encoding
I’ll import these, though from os I’ll only get a few things.
Basic import of a JSON file
import jsonimport chardetimport pandas as pdimport os
Once I’ve done this, I can use these modules in my code.
Now for the code. The first thing is to find my files. I’ve stored json files in a “raw” folder, which I assume is below the place where I’m running the code. In this case, I have two files.
Here’s a little setup code that sets the path (which could be an argument to the file), but creates a path to the files and starts a loop:
mypath = '.\\raw'onlyfiles = [f for f in os.listdir(mypath) if os.path.isfile(os.path.join(mypath, f)) and f.endswith('.json') ]# loop through the filesfor f in onlyfiles:
In Python, once I want to create a set of code in a loop, I need to indent it, so the next few lines are indented below the for statement above. I’ll repeat that for clarity.
In the loop, I want to do a few things. First, I get a path to the file with the os.path.join command, which builds me a path that works in various functions. Next, I want to use the chardet module to detect the encoding of the file. They should all be the same, but I had some issues when expecting the default encoding (this post helped). This ensures I get the correct encoding for the file.
Lastly, I’ll open the file.
```
loop through the filesfor f in onlyfiles: currentfile = os.path.join(mypath,f) enc=chardet.detect(open(currentfile,'rb').read())['encoding']with open(currentfile,'r', encoding=enc) as json_data:
``` Again, I have a with statement and want to run other code, so I’ll indent the next line. The next line reads in the file using the json module, which has a load() function. This function knows how to parse a json file so that we can work with it.
Lastly, outside of the with command, I’ll return to the for loop be de-indenting one level and calling a pandas function to take a portion of the JSON file and load it into a dataframe. Think of a dataframe like a resultset in SQL or a datatable in C#. In this case, I’ll take the “sessions” structure. I print the first five rows with the head() function.
The entire code structure looks like this:
```
Basic import of a JSON fileimport jsonimport chardetimport pandas as pdimport osmypath = '.\raw'onlyfiles = [f for f in os.listdir(mypath) if os.path.isfile(os.path.join(mypath, f)) and f.endswith('.json') ]# loop through the filesfor f in onlyfiles: currentfile = os.path.join(mypath,f) enc=chardet.detect(open(currentfile,'rb').read())['encoding'] with open(currentfile,'r', encoding=enc) as json_data: data = json.load(json_data) # get session data from json df = pd.DataFrame(data['sessions']) # print the head print(df.head())
``` The results look like the image below. I’m not covering how to run Python or anything else, but you can see the first five rows with the session title and a couple other elements.
The raw JSON looks like this for the first file with the sessions element.
Now I can work with the data and query, transform, rewrite, store in a database, whatever. I’ll cover how to move this data in another post.
SQL New BloggerThis post took me about 15 minutes to write, mostly because of looking up some links. The code itself was for something I was already doing, so after getting the code working, I wrote this post using it.
This is a good example of something you could write to show that you are building some data warehouse skills, which are valuable for many employers.
It seems that when I travel to offices these days, it’s standard for most desk setups to have two monitors. I think all the desks at Redgate have a docking station and two monitors for people to use. They also convert to standing desks, which is handy. I have a standing desk that I use regularly, and it’s nice to have that option when I visit an office. At a number of customer sites, I’ve seen similar setups, sometimes with laptop/monitor lifts instead of desks that rise.
Recently I saw a docking station announced that can support four monitors. I wonder how many of you want, need, or use more than two monitors. While there is often a standard in offices, since many of us work part or full-time at home, perhaps you have a different setup. Maybe you have one or two large monitors instead of 3 or 4 smaller ones. I’d certainly be interested to know if any of you have more than 4 monitors.
I run three 24″ monitors on my desk. When I go to an office and use a docking station, I can use my laptop as a third, though the resolution gets a little funny. If I were there for any length of time, I might try to reconfigure the laptop to work with the lid closed. Maybe that’s a chore for the next trip. At home, however, I usually have SQL Server Central and writing tools on one monitor, Chrome with lots of tabs as my main workspace for research and general work, and Outlook/Spotify on a third monitor. That lets me focus on one monitor most of the time glancing at another one for reference or copy/paste work.
I tried four monitors in a 2×2 configuration at one point, but I didn’t like looking up and down. Even now, I often really use two monitors, with side glances at a third one at times. I have been tempted to get 2 larger monitors (27-30″ range), but I’m not sure I would be more productive and not sure it’s worth spending the money for no real benefit I can perceive.
I’ve seen some neat setups in people’s offices. I see some developers using vertical monitors, though I haven’t found that to be ergonomic for me. I seem to have too many long lines of code or text. I see some people using a laptop with one monitor, and I’ve seen some friends who are creative with videos/live streaming that use more than four monitors.
I don’t know I have met anyone that has a gaming chair/monitor setup, but I’m sure someone out there has bought or built something like that. If you have, or you think you’ve got a cool setup, let me know.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
ioia – n.the wish that you could see statistics overlaid on every person you encounter – checking the signal strength of their compatibility, a measure of their trustworthiness.
I subscribe to the NFL Plus mobile package every year and watch various games on my phone/tablet as I have time. Often I watch some while working on the ranch, cutting grass, shopping in the grocery, etc. I bring this up as there are a few viewing options in the package. I can watch the normal game broadcast, but I can also see one with stats. This looks similar to the image below:
I thought I would enjoy this, but I don’t. I don’t mind some stats at times, but I don’t like when there is an overlay while the players are in action.
The idea of ioia being always overlaid on what I see, ala the Apple Vision or Google Glass views, would be annoying, and distracting. Already people checking mobiles is annoying.
If I could control the display, and dismiss it, then maybe ioia is something I’d embrace. Of course, there’s still the concern with data quality as a lot of ratings for people are very subjective.
From the Dictionary of Obscure Sorrows
I love Chicago. I went to visit three times in 2023: a Redgate event, a volleyball tournament, and a wedding. Each time was a lot of fun and I look forward to coming back at the end of this month.
The Redgate Summit is coming to Chicago on May 29. Three weeks from today! Register today and save your spot. If you get on the wait list, reach out to your account executive as they have their own tickets to give away.
We’ve had two amazing Summits in Atlanta and London and we’re bringing the show to the Windy City. We’ll be covering a wide variety of topics related to databases, across three tracks.
- New and Future Tech – leveling up skills about database DevOps and teams
- Deep Dive Solutions – technical talks on innovative strategies for delivering software
- Leadership – focused on strategic initiatives
We have something for everyone. Technical people, managers, senior leadership, and others. Tell your colleagues and come as a group, taking in different tracks and then discussing them alter.
I hope to see you there and register for the Redgate Summit in Chicago today.
I’ve got a few certifications and quite a few more that have expired or aren’t relevant. Does anyone think Windows NT 4.0 or SQL Server 6.5 matter? If you need help in those areas, ask someone else. Unless you have a crazy budget with a willingness to pay a ridiculous hourly rate.
Kamil Nowinski had a recent video discussing why IT certifications are still relevant. He had ten reasons, and if you want to watch the entire show, you’ll hear his reasons and some rationale why he thinks they matter. It’s a good set of reasons: keeping up with tech, practicing learning, demonstrating a commitment to some technology, finding a community of certified colleagues, and more.
In some sense, many of these are tightly coupled, but still worth pointing out. I think for smart, driven, ambitious people, certification is something that you can take advantage of to add to the learning you’re likely already doing: sharpening your saw. That’s an analogy to the craftsman that keeps his/her/their tools ready to go. For us in technology, this is usually our mental acuity with software (or perhaps typing).
For those who aren’t motivated, getting a certification just to add letters after your name devalues the certification for sure. We saw a lot of paper-CNEs (Novell), paper-MCSEs (Microsoft), and paper-CNAs (Cisco) in the 90s. Poor interviewing techniques combined with a glut of certified-but-not capable staffers soured many people on certifications. For good reasons.
However, when you interview someone with a certification, or if you want to get one, pursuing a certification using Kamil’s list means you are driving your knowledge forward in a focused, organized fashion. The certification isn’t the goal, but a step on the way to becoming a more capable technical professional.
I got a couple of certifications in the last few years, and I felt they helped me grow and learn. I didn’t always use all the skills directly, but I learned things that have helped me in my job and I didn’t regret the time spent. I might work on another one in 2024, but time is always in short supply, and I’ve got other things to learn.
Even if I’m not trying to get certified, I continue to learn for many of Kamil’s reasons, just without an examination during my journey.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
One of the things that I’ve found with cloud computing services is that the people involved with managing these resources sometimes get asked to become financial accountants.
I saw an interesting post from SQL Rod asking about this new task as something all of us might need to consider a part of our job in the modern world. He asks if we are Techouncants or Accountechs? This isn’t in the sense of being the Financial DBA trying to manage costs in the cloud, though that is part of his post. It’s more about making smart financial decisions. Certainly, as more workloads move to the cloud, and they are for many of us, we likely need to keep an eye on costs, usage, and tuning.
However, there are other choices. When I had to spec servers, we wanted to get something with room to grow, but not the top-of-the-line most expensive option. I wanted something engineered well for the next few years, and certainly not under-engineered. Usually because adding resources was hard in those days.
Virtualization has made things easier, but often there are still some limits. Still, the goal is matching resources to workload with room to burst if needed. At the same time, are there other choices we can make? Rod asks about choosing the appropriate edition, the HA/DR strategy, tooling, and more. Can we be efficient with our use of money, time, and results, trying to neither overengineer nor under-engineer?
The whole post has a lot of “it depends” in it, as the decisions on what you should choose are something you have to think about and arrive at a decision that balances costs with other factors. It can be clearer often to choose one thing over another, but it’s not always crystal clear which choice is better. I like his hardware analogy in the post. I’m not a fan of just throwing hardware at a situation, but in some cases, that’s the best solution.
The cloud makes it easier to see where costs are. As a Redgate customer put it recently, “tuning queries becomes more important in the cloud as we can see exactly what each one costs.” That can be true for your SQL Server VMs, but it’s often not seen as important when the costs are a one-time spend for your hardware. However, Jeff Moden might argue that you should be tuning queries even without chargebacks or a cloud bill. It will save time for your users, which is always valuable. I agree, since I think we have no shortage of work and time is our most valuable resource. Save it when you can for the most people.
That often means a big part of being a DBAccountant is being able to show, lead, or somehow get others to write better code early. Ensure everyone (or automated tests) knows how to view execution plans, learns to use them, and tries to write more efficient code the first time.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
I’ve been gone from home for two weeks, though I’ve only been in Australia for about 12 days. I lost a day in flight going across the date line. I’ll get that back later this week as I’ll actually land in LA a few hours before I left Melbourne. I leave Fri morning at 930a and arrive in LA Fri morning at 640a. That’s a crazy time travel situation.
In any case, I’ve had a good time here. My wife and I spent a week doing some touristy stuff. The iconic shot with the Opera House was first, along with some travels around the area and a nice relaxing week.
I went to Brisbane for the Redgate Seminar, and didn’t have much time there, but I came back to Sydney for the weekend. Unfortunately I was alone as my wife had to head home to get back to work on the ranch. I took a little time at the Sydney art Museum where I still couldn’t get away from horses.
I also saw a neat modern art repurpose of old tube televisions as a cello.
Lots of other interesting artwork there as well. I’m sure my kids were both jealous and annoyed with a lot of SMS pictures of my experience. I’m not usually a big art person, but it was raining and my daughter (a fine arts major in uni) has influenced me a bit, so I took a few hours to try and see the world from her perspective.
I also spent a few hours Sunday at the theater. I’d seen the billboards with my wife and I loved the movie as a kid, so I decided to try the musical. It was good, and I enjoyed the singing and the way they moved scenes (and songs) around to work on stage. Great performances, but I still like the movie better.
I’m also heading to Melbourne later this week, but I’m only there for about 45 hours, so not a lot of time to see anything. I spent a week there in 2019 and loved it. I hope to get back there with my wife sometime, perhaps on the way to Tasmania or Adelaide.
Now, back to work before the Redgate Seminar tomorrow.
For most of us working in technology, I think we understand that if something is broken we might need to work. Not that we have to, or we need to, but we might need to. Perhaps you feel differently, or your company approaches on-call in another way. If so, let me know today how you deal with staff being on-call.
In my career, there are jobs with formal on-call, informal on-call, or even no on-call. In the latter situation, there isn’t anyone who is prepared to handle issues outside of normal working hours, but that doesn’t mean if management calls you can ignore them. It’s that the organization didn’t expect issues. I worked in a small company (< 50 people), where we primarily had systems for people who worked in the office, and nothing was running at night (outside of backups). Normally no one knew if there was an issue overnight or on weekends, but I did get called by the owner when he went in one weekend and couldn’t receive a fax on our computer system. So I guess I was the emergency-on-call person.
In both large and small companies, I’ve had formal on-call situations where humans or automated systems would know I was assigned as the person to contact for issues. These might be 24-hour periods, weeks, or even a month at a time. In informal situations, usually, there was a list of technical people’s contact information and anyone needing help would just pick one and call, often based on who they thought could solve the problem.
For most of my career, being on-call hasn’t been a large burden. I’ve been interrupted during some events, dinners, or holidays, but relatively rarely. My wife and I have felt it wasn’t too burdensome, especially given compensation and other flexibility. At the same time, there have been some places where on-call is a disruption to my life. Usually, I’ve tried to leave those positions as quickly as I could, though that might take months.
I always ask about on-call responsibilities when I interview for positions. It’s not that the answer is a deciding factor, but it is a factor. If on-call is more demanding, I would ask for something in return: perhaps comp time, more salary, or some other flexibility.
What is your on-call situation? Is that acceptable to you? Have you never been called outside of working hours? Let me know today.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
mcfeely – adj. inexplicably moved by predictable and well-worn sentiments, even if they are trite or obvious or being broadcast blindly to the masses.
I get mcfeely all the time. Recently I was listening to A Beautiful Life, from a movie by the same name. It’s a simple song, simple lyrics, but it creates a lot of emotion in me. It brings back memories of finding out my wife was pregnant with my daughter, of the early years when I put her to bed and read stories, of when she grew up and moved away for college.
Music often does this for me, as do movies/television or even a well written passage in a book. It could be joy, sadness, anger, or anything else. I enjoy the emotion of the situations I’ve escaped into.
From the Dictionary of Obscure Sorrows
I wrote a post recently about pruning branches in git. That’s part of the job, but the other part is removing local branches. This post looks at one way to do that in a semi-manual fashion.
This could be automated, but it took seconds, so I did a quick manual thing. I’ll work on an automated way, but since I do this rarely, manual is fine for me.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. You can see all posts on Git as well.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
Getting a List of Branches to DeleteIn the last post I showed how to get a list of branches with dry run. This was the image I showed of branches. If I re-run that without the dry run, the branches are removed.
The output is similar, but either set of output works. Once we have a list of branches, what do we do? Let’s use SSMS to help.
SSMS Makes This EasyIf I highlight the results of my git prune, I can copy/paste those into SSMS. You can see below I’ve done this, and then held the ALT+Shift key to select a bunch of text in a box. This is all the text apart from the branch names.
Once I’ve done this, I can let go of those keys and type “git branch –d “, which will replace the text on every line. You can see this below.
This is a great technique, and while it works in VSCode and some other editors, I usually have SSMS open and it works very well here. I then select all this text, paste it back into the CMD window, accept the note that this is a multi-line paste, and all my deletes run.
Voila, all remote branches deleted are removed from my local git install. A few of these were already removed manually as I experimented.
SQL New BloggerAs I mentioned in the previous post, version control skills (especially git) are core for most technology pros. DBA, developers, sysadmin, anyone working with modern software development or administration likely needs to know about version control.
This post was about 10 minutes. You could write this in 15 and showcase your tech skills to a future employer.
I saw a quote recently that resonated with me. It’s not something I’ve often struggled with, but I have at times. Here’s the quote:
“Life rewards action, not intelligence. Many brilliant people talk themselves out of getting started, and being smart doesn’t help very much without the courage to act. You can’t win if you’re not in the game.” – @JamesClear
Part of my goals in life are to help others and motivate them to engage in life and drive themselves forward. I have tried to motivate my children, I work with kids I coach to grow and change, and I (think) write about this often here. I want you to take charge of your career and life and drive it forward in the way that works for you.
However, you have to work at things. The quote above notes that it’s not how smart you are, but having success in life comes from action. From doing something. From making efforts. The last part is the most important, if you don’t engage and become part of your game, your life, you can’t win or achieve your goals.
As Ferris Bueller said, life can move pretty and you can miss it if you’re not paying attention. As I get older, that means more to me. Life slips by, days, weeks, months, even years seem to fly by. Especially after the pandemic, where I see someone and realize it hasn’t been a year or two since I’ve seen them but four years.
You don’t have to be extremely driven and type A. I think I might be a little too driven at times, living in chaos, but I met someone recently who spent more than 5 years trying to transform a company. This individual worked their job, spent nights and weekends trying to learn more about their business, and drove themselves and others well beyond what I’d consider a balance in life. They were working well over 60 hours a week for years, as were some employees. I don’t think that’s a great way to live, and whether it’s work, hobbies, or something else, forgetting to balance the various parts of your life isn’t good.
At the same time, inaction, not trying to grow, learn, practice, or even attempt something new in any part of your life isn’t good either.
Work on your career and skills but balance that with the time you need away from work. Invest in your education to change careers if you like, but don’t plan on working what is essentially two jobs to become a way of life. At least not for too many years. Enjoy your hobbies, but remember there is family at home that deserves and needs your time as well. Find ways to engage in life, but in a way that respects all the commitments to work, family, friends, faith, hobbies, and yourself. Especially don’t forget about taking care of yourself.
Effort is often more important than the short-term results. Just being engaged and in the game gives you a chance of success, but more, it allows you to enjoy the ride. Just do so in a balanced way.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
On Thursday, May 2, we have the Redgate Seminar in Brisbane. I’ll be at there along with fellow Redgate engineers, account executives, Octopus Deploy engineers, fellow Microsoft MVPs, and lots of data professionals.
Come join me and we’ll have a great day talking about End to End Database DevOps across a number of sessions and discussion panels.
Don’t miss the day and register today.
the standard blues– n. the dispiriting awareness that the twists and turns of your life feel new and profound, but are not unique – marked by the same coming-of-age struggles as millions of others, the same career setbacks, the same family strife, the same learning curve of parenting – which makes even your toughest challenges feel harmless and predictable, just another rename of the same old story.
In many ways I relate to the standard blues, though I don’t find them blue, but comforting. I was at dinner with our volleyball team and their parents. One of the other sets of parents had a 4-year old, who struggled to sit still and made dinner a challenge.
I remember those days. Remember feeling this is way harder for me and my wife than others. Especially when we had 2 little ones, and everyone else didn’t. However, over time I realized that it was new and hard to me, but it’s the same thing others went through. I still see that today with parents on that part of their journey.
I think a lot of things that feel tough in life, stress over finding a job, over your first review, over purchasing a house or car, are not that tough a challenge as they feel in that moment. They are just a new experience for you.
Even when my father passed, it was a shock, but I took comfort from others that had gone through the same thing, and from the knowledge that it’s a part of life.
Sometimes events are blue, and sometimes they are just new events. Fortunately most of the time they aren’t as unique or hard over time as they are in the moment.
From the Dictionary of Obscure Sorrows
It’s Wednesday. However, it feels like Tuesday. I left Denver about 25 hours ago to fly to LA and then Sydney. It feels like one day, but by crossing the international date line, I lost a day.
Don’t worry, I’ll get it back when I come home in a couple weeks.
Today starts vacation for my wife and I. We’ll enjoy some time in Sydney and Brisbane before I go to work next week.
No much blogging until then, apart from a new word Friday
I leave tonight for Australia. I was in London 3 days ago, so this will complete my halfway around the world trip when I time travel tomorrow and skip Tuesday. I’ll arrive in Sydney on Wednesday morning.
I am very lucky and I appreciate the chance to go back down under for work. I have a few seminars I’d delivering with fellow Redgaters, MVPs, industry experts, and Octopus Deploy engineers. I’ll be busy these days:
- May 2- Brisbane
- May 7 – Sydney
- May 9 – Melbourne
That’s next week, but I’m heading down early to recharge a bit. My wife is traveling with me to Sydney, where we’ll spend a few days later this week. She’s never been to that city, and I have literally been there for about 53 hours. We’re looking forward to some sightseeing, hiking around, and enjoying the Australian hospitality.
We’ll also take a few days in Brisbane before I go to work, checking out the Gold Coast and seeing a few friends. I’m looking forward to going back, as I enjoyed 4 days there before the pandemic.
This should be a good break from work before a busy week. I’ll be lightly blogging a few things, but mostly enjoying this break on the other side of the world.
Last year, I read Surrender, a book by U2 lead singer, Bono. Bill Gates listed this as one of the top books to read at one point, so I picked it up and dove in. I have enjoyed U2s music since I was in high school, and was interested to hear what made Bill Gates recommend his book. The book is partially a journey of U2, but mostly a look at how Bono’s view of the world and life has changed over time.
Bono grew beyond music in his life to become an activist and try to shape the world into a better place. Whether you agree with his efforts or focus or not, it’s admirable that he has tried to be more than a rich and famous singer. He’s had to build more skills around how to communicate with others, convince them to take a course of action, and educate himself about the world. In trying to build these skills, he’s founded or worked in organizations around his time with U2.
As a part of that, he had a great quote about leaders who are busy elsewhere but try to be involved in different parts of the business. He called this seagull management, and it was something he tried to avoid doing as he only comes into the office periodically.
Seagull management is where you fly into the office, shit over what everyone is doing, and fly off again. I wonder how many people in management do this but think that they are motivating, helping, or improving their workers’ efforts. Trying to bring their view, experience, knowledge, etc. to others, but invariably doing so in a way that doesn’t resonate with workers. Perhaps it’s not even helpful if management hasn’t taken the time to understand why people are working in a certain way.
I also see this with technical leads or senior engineers who come into a situation, often with strong opinions. They might express how they would have coded or architected something completely differently. Perhaps even informing existing staff that they are doing something wrong. Whether good intentioned or not, seagull management doesn’t help improve any situation.
We make bad decisions, we may build something without considering all the information, or perhaps the situation changes. We all find ourselves in situations where the technology doesn’t seem to be well matched to the environment. We might wish things were different. However, no matter how we arrive at our present situation, we are there. Extracting ourselves from any legacy environment takes time and has to be a journey that is undertaken with support from both technical staff and management.
Most of us want to build great systems that work well for our clients and are admired by others. We rarely find ourselves in a place where we have the time and resources to do that. We can refactor, evolve, and grow our systems to be better, but it does take time. We need a goal, direction, support, and understanding that change is a journey, not something that a manager can fix on their rare visits to our environment.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
apolytus– n. the moment you realize you are changing as a person, finally outgrowing your old problems like a reptile shedding its skin, already able to twist back around and chuckle at this weirdly antiquated caricature of yourself that will soon come off completely.
I don’t know if I am changing that much, though hopefully I’m still maturing a bit. More, I think about my daughter, who graduates college this year. She’s changed so much in the five years since she left home for college, outgrowing the problems and concerns she had as an 18-year old.
I think I went through that myself, leaving home at 17 for college, coming back 4 years later a completely different person, viewing the world completely differently.
From the Dictionary of Obscure Sorrows
As I’ve been working with SQL Saturday and managing changes to events, I’ve accumulated a lot of branches. Even though I’m a solo developer, I decided to use branches, as I expect others to share this load in the future. This post looks at how to start cleaning those up.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. You can see all posts on Git as well.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
Finding Old BranchesWhen I ran the git branch command, I saw this. There are a lot of old branches in there.
I decided that I should reduce this number. After all, even removing stale branches means I have a lot of events in flight.
We use GitHub, and when I go on the site, I see lots of branches, some of which date back to last year. Those events are done, so I decided to delete some branches. In the image below, there are three branches. To the right, there are delete icons on the bottom two as I’ve already pressed the one to delete the remote branch on the top one.
Now, how do I delete the local branch? Let’s start by removing it.
Git PruneThere is a command to remove references to remote branches that are deleted: git prune. I deleted a few older branches, and then ran git prune for remotes, with the –dry-run option. This tells me what would happen. As you can see, a number of branch references would be deleted.
Nothing in here I’m worried about or that is active. I’ve deleted these on GitHub, so I’ll re-run the command without dry run. This removes the references.
Unfortunately, the local branches still exist. We don’t remove these, as it’s possible I have work on a local branch not sent to the remote, so doing this automatically, even if I do it, is dangerous.
I’ll do another post on removing the local branches.
Automating the Removal of Remote referencesI might want to remove local references for branches that get deleted on the remote. This is useful if you delete branches on merge. I don’t in this case, as I’m often using the same branch for multiple changes for an event, rather than a new branch for every one.
One way to do this is to change the config with this:
git config remote.origin.prune true
This will then run the prune on each fetch or full. This helps keep things cleaner, though local branches still exist. However, if I commit to a local branch and push, I’ll get an error that I need to configure the upstream. That helps with me being aware of what’s active or not.
SQL New BloggerUsing version control is a core skill for anyone in technology. Even database people. You could write posts on how you use or learn about git (or something else) and showcase your skills.
This post took me about 15 minutes to write, even with screen shots.
Recently I traveled to visit a customer who has an in-the-office culture. They have multiple large buildings outside a major US city and almost all their employees (7000+) live nearby and are expected to be in the office the whole week. More senior people can opt for 4 10-hour shifts rather than 5 8-hour shifts, but with few exceptions, they have people in the office.
I hadn’t seen that in a long time. Almost every customer is mostly remote or some level of hybrid (usually 2-3 days a week in the office). What’s more, they have an open culture, with rows of desks for teams and spaces between the rows for managers and directors. No cubes!
Everyone below a senior VP/C-level is at a desk. There are conference rooms and smaller quiet pods, but mostly everyone works in wide open spaces. From developers to sales to finance to human resources. It’s both chaotic and loud, but also refreshing. I also loved seeing so many people wearing company logos on t-shirts, jackets, hats, and more.
The environment reminds me of my time at JD Edwards. We had cubicles, but mostly open ones with half walls to the side and one full wall at the back where two cubes met. However, it was a very strong culture of comradery and closeness that I hadn’t seen at any large company before or since: until now. Teams might be close in some places I’ve worked, but the entire company at JD Edwards felt like a family of people who were working together. We didn’t always get along and argued, but we were all united. It was a great feeling.
I miss that. Redgate is a great company, and I enjoy my co-workers, but I miss going to an office and seeing everyone inside it. Most people come to the office once or twice a week, but I miss my visits when most everyone was there every day. I’ve worked remotely for over twenty years, but I do still miss the office some days. Certainly my visits to the various Redgate offices, while good, have me wishing I saw more people there every day. I’m heading there tomorrow, and I hope I see lots of people.
I don’t know that mandating everyone in the office is a better choice, but I do know there are good things about gathering groups of people together. Strong leadership, empathy, a clear vision, and a balanced workload are important for any organization, but especially important if you want everyone in the office these days. This customer did a great job with it, and I think they would still have a strong culture with remote work, but their choice works, and I’m a little jealous.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Yesterday was a long day in London. I arrived late in the am, and as I was walking from the plane to the border check, I went down the escalator on the entering-the-country side of the terminal. I had this view:
There’s nothing special here, but through the glass there are the outbound escalators that I’ve taken up and a Caffe Nero coffee shop. That’s a place I used to stop often on my way out of the UK early in my time at Redgate.
I used to always stay at a hotel near the airport, first the Renaissance Hotel with Brad McGehee and later at the Hilton or DoubleTree with Grant. I’d often have an early flight, which meant getting up and trying to catch the Hoppa bus or get a cab. The hotels usually didn’t have breakfast ready then, so I’d stress over the transport, which could be funny, get to the airport and through security and then stop here for a cup of coffee and a chocolate croissant.
Hectic, but very impressionable trips on me. So many memories of sitting near a gate, eating and looking forward to coming home. I didn’t travel much those early days outside of a few UK trips, and it was a relief to be going home.
The last 5-6 years, with a direct flight to Denver and more airline status, I get a late flight, enjoy the lounge, and skip Nero, but I always reflect on my memories there as I think of the early days, paying with coins or bills, or swiping my American credit card and having to sign, something so few European travelers did.
Hotel ConfusionI spent the night in the Hilton at T4, which is one of two hotels I use now. There is a Garden Inn at T2 as well, which I often pick when leaving since I can walk to the terminal. Yesterday, arriving tired in the morning, I was on autopilot and walked to the Garden Inn. They said they didn’t have a reservation for me.
Doh! I hadn’t even checked, assuming I was there, but when I looked in the app, I realized I was at the T4 Hilton, which is a nicer hotel, just a bit of a trek. I walked back, found a train and was at the Hilton 30-45 minutes later. Not a long time, but also not quick for a tired traveler.
A nap, some work, and a workout made me feel more human, but still a long day. Plus I had a 730p call in Denver, which was 230a in London. It’s been a long 24 hours.
However, I had a quiet dinner in the hotel and snapped this picture after.
This large atrium has a lot of memories for me. I’ve been here often, including quite a few times with my wife. We’ve eaten in this spot, grabbing dinner after a trip in the EU somewhere, having a quiet night before a morning flight. While this hotel is a walk and train to T2, in the morning, it’s an early morning and stressful, so an early night here is often on our agenda.
This hotel also has a special place in my mind as the gym is on the ground level (this is the first floor), but it’s on the side of the hotel with no windows. It’s a dark, quiet spot, and it’s where I once got up at 430a to run.
I had a long running streak (day 1000 here) where I ran at least a mile every day. I did this for 1564 days, over 4 years. During that time I had a few trips to the UK and back, with the return trips back being hard. Depending on weather and equipment, I’ve had 20+ hour trips. Most of the time I’d get home and manage a jog at home, though one time at 1130p at night. Rather than chance things, I got up one time in this hotel, the only person on a treadmill at 435a trying to get a mile done before heading to the airport.
That sticks with me.
Today starts a wild stretch of travel for me. I land in London today, having left the US after coaching the final volleyball tournament of the season yesterday. I’ll spend the week in London for the Redgate Summit, with a quick trip to the office for a day to see some colleagues. I return home Friday and then leave Monday for Australia.
That doesn’t sound too crazy, but I’ll be back in London the first week of June and on the road most of the time until then. My schedule:
- Apr 15-19 – London
- Apr 19-22 – Denver
- Apr 22-May 10 – Australia
- May 10-May 13 – Denver
- May 14-20 – New York
- May 20-28 – Denver
- May 28-30 – Chicago
- May 30-Jun 1 – Denver
- Jun 1 – June 15 – UK and EU
Then I’m home. At least I think I’m home for a few weeks. Nothing planned yet, but things might change.
It’s not all work. My wife and I will take about 8 days to of holiday in AUS, and another 6 in the EU. The NY trip is to see my daughter graduate college.
Overall, it’s a wild time, it will be hard, but also very exciting.
My plan is to rest regularly and work out regularly, likely with walking and biking so I’m not overloading myself, but not sitting still either. I’ll be looking to drink water and eat lots of salads and proteins as well, trying to limit the carbs, though I’m sure I’ll indulge a few times. I’m also packing lots of vitamins to try and keep myself healthy through this stretch.
Hopefully I don’t drop any balls along the way.
BTW, think this thing will get from Denver to the UK?
As I work through 2024, I found myself doing a little more vacation planning this year than in previous ones. In 2022 I traveled quite a bit, but my wife went with me often. We went to Europe 5 times that year and added quite a few vacation days around my work trips. My wife thought that was a great year.
Last year, 2023, was different. I traveled more (36 trips), with most of them being short. When I traveled that much, I wanted to end trips quickly and get back home. I learned that was too many, and also too disruptive for life. I got behind on things I needed to do at home, my wife went with me less because many trips were all work, and I lacked energy from the pace of moving all over the world.
As a result, my boss and I are more closely watching my travel schedule, and I’m consciously working to ensure I take some breaks between trips. Part of that is doing some planning. So far this has me reducing the number of trips (10 in H1), but also including 3 good-sized vacations away from work during that time.
Note: don’t feel too sorry for my travel load. I’ll get a holiday each in the US, Europe, and Australia.
I know a lot of people like to schedule their vacations at similar times each year. The end-of-year holidays are often a time when many people travel, but I know some people who take time every June, others every August, often corresponding with school breaks for children, family reunions, or some other event.
When do you like to take your vacation? Are you a many-long-weekends-through-the-year person? Do you go on one long trip a year? Love holidays to get away or stay home and work because work is quiet?
I don’t know if it matters, but I’m always interested in what others do and why. I prefer more, shorter vacations, but convince me why your long trip is better. Tell me the amazing things/places you’ve done/seen. Maybe I’ll get some ideas for a future trip.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
I saw someone struggling with getting started with a Visual Studio project and Azure DevOps. They got a conflict, which I’ll show and then get you started with an empty repo. Another post for me that is simple and hopefully … Continue reading →
I’m doing a webinar next week with Bob Ward, a principal architect for Microsoft working on Azure and SQL Server. I’ve had the chance to work with Bob quite a few times over the years, and I’m honored that I … Continue reading →
Data warehousing has changed a lot in my 30+ years of working with data. With Microsoft Fabric, this editorial might be more interesting to read and reflect on how the world has changed.
I’m on holiday today with my wife in Amsterdam, so you get to read: The New Data Warehouse Choice
It’s a day off for me, so why am I blogging?
I actually wrote this a couple weeks ago as I was prepping for this trip. I’m in Amsterdam today, taking a few days with my wife.
I came over to the UK on 1 Sep and I won’t go home until 16 Sept. With a very long trip, I decided to break this up in the middle. My wife joined me on the 7th, and she will go back on the 14th. It’s a vacation for us both from work, and taking advantage of the work trip means that I don’t feel quite so worn out from a long trip.
If I’d stuck around this week in Cambridge and worked, which was something I thought about, I’d be pretty tired when I went home. Plus, I would really be missing my wife. We are rarely apart for more than a week, and it’s hard when we are.
Not everyone travels for work, and many of us have other family commitments. However, my wife and I are always looking for opportunities to travel, and when things line up with work, we try to take advantage of them.
Hopefully your day is going as well as mine.
vulture shock – n. the nagging sense that no matter how many days you spend in a foreign country, you never quite manage to step foot in it – instead floating high above the culture like a diver over a reef, too dazzled by its exotic quirks to notice its problems and complexities and banalities, while drawing from the heavy tank of assumptions that you carry on your back wherever you go.
I guess not technically a word, but two. However, as someone that has enjoyed and loves traveling, this hits home.
My wife and I have made it a point to visit other countries, aiming to see a new one each year. In the last five years, despite the pandemic, we managed 11 countries outside the US, perhaps 5 new ones.
As we travel, we’ve tried not to do too many tourist things. We do a few, but we’ve enjoyed renting an AirBnb in a neighborhood and living for a week like we might if we resided there. It’s been neat to walk around, to shop in groceries, to experience life in small, quiet sections of cities and towns.
At the same time, I often feel foreign. I’ve grown up as an American for most of my life, and its culture is very ingrained in me. My habits, my languages, my preferences, my food, it’s very colored by my US upbringing. While I do try to experiment and try new things, everything I do is compared against my American-centric thoughts.
An example. In Seattle recently, we went to a Filipino diner for breakfast, where I had Sinangag and Longganisa with eggs. I enjoyed it, and I need to make some, but it was strange not to have a more American breakfast, to not have something sweet, to not have a more salty sausage. I was in the culture, but not in it.
I’ve felt the same thing in Greece, in France, in Portugal, in India, and other places. Even i the UK I feel somewhat immersed in and outside of the culture. Floating above it like a diver over a reef.
From the Dictionary of Obscure Sorrows
Brent wrote about what a good DBA looks like and challenged people to write a testimonial for them. There are some great comments on the post, and some funny ones. It’s worth a read when you need a break from other work. I especially chuckled at the picture Brent used. I think ours was better.
That got me thinking: what do I want in a team? Or maybe, who do I want in a team?
I have a great team now, but we are fairly distributed and work independently. I have great co-workers at Redgate, though I don’t often work that closely with any of them. I work often with some of them, for specific things, but usually it’s coordination rather than the tight, back and forth I’ve often had with technical teams.
I realized at some point in my career that I was often being asked to do the same work. I needed to manage systems. I needed to write and tune queries (or rewrite those for others). I needed to manage security. Most importantly, I needed to find solutions to the constant set of questions, problems, and demands from others in my organization. I learned how to teach myself things, how to research, how to test, and how to talk to others. I became good at clarifying what people needed and then finding a way to meet those needs. I think I turned into part of the Incredible DBA Team, even though often it was a team of just me.
The characteristics Brent talked about were important to me.
A little. I have always felt that I could work with others if they want to learn from me and teach me things. They have to want to collaborate and get things done as a team. That led me to worry more about who I worked with than what the job was. If the compensation was good, then the decision to take a job often came down to who would I work with. After all, the work was often very similar.
Think about your situation. What do you want to see in your coworkers? I’m sure you want people that you get along with and carry their weight. Maybe you want to learn from them. Maybe you want to be able to teach them. Maybe you have specific skills you wish were stronger on your team.
Leave a comment and let us know what your ideal team looks like. As always, if you want to leave an anonymous comment, send me a PM on the site with what you’d like posted in the discussion.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
The Future Data Driven 2023 virtual conference is coming on September 27. Register today and save a note in your calendar.
I have been honored to speak at this event in the past. I’m not speaking this year as I’ve been buried with work and travel and need a break. However, I see a great schedule (scroll down) with some sessions that I will try to watch. A few on my mind:
- PowerBI, DirectQuery and SQL Server. It is a good choice? – I don’t know a lot about the differences, so I’m hoping to learn something here.
- Getting Started With Governance For Your Power BI Estate – Outside the US, I have customers asking about this, so I need to learn what options are out there.
- Azure Synapse link for SQL: what, why and how – I want to know more about this and how it works.
I can get information in other ways, but a conference is a good chance to hear what someone else has learned and shares with me. Even if I listen in the background, I’m learning some new things.
There are lots of other great sessions, so register today.
I ran across an interesting gotcha while trying to run a Flyway command from PowerShell. Specifically, the snapshot command and providing a parameter that is the snapshot.filename parameter.
Someone reported an issue with Snapshot, so I tried it from a command line. It worked fine for me, but a sharp Redgate developer noted that the person was running something like this from PowerShell:
flyway snapshot -snapshot.filename="deployed1.snapshot" -url="jdbc:sqlserver://localhost;databaseName=FWSimpleTalk\_1\_Dev;encrypt=true;integratedSecurity=true;trustServerCertificate=true"
As you can see below, this caused an error.
I checked my version. Surely “flyway snapshot” was valid, and it was.
However, there is an issue here with PowerShell and the parameter. In this case, the PoSh will try to interpret this as
The issue is lightly explained in this SO question, though I can’t quite figure out where this is documented. However, using quotes around the parameter name fixes this. Note, I switched the parameter value to single quotes.
flyway snapshot "-snapshot.filename='deployed1.snapshot'" -url="jdbc:sqlserver://localhost;databaseName=FWSimpleTalk\_1\_Dev;encrypt=true;integratedSecurity=true;trustServerCertificate=true"
As you can see, this works.
PowerShell is closed to command shell, but not the same. Keep that in mind as you build automation. Be wary of how your parameter values might be interpreted and test lots of values.
The economy might be good or bad for you right now. Some of that depends on where you live, what your employment situation is like, what your habits dictate about how you live life, and more. No matter what your situation, likely there are people around you that complain about the world and others who think things are fine. There are likely more of the former than the latter, but that’s because humans tend to complain out loud more than they praise.
When people think there is an economic downtown for themselves, they may be more likely to engage in malicious activities. While I don’t think most data professionals will start to hack other systems, or even their own employer’s systems, there is evidence to support the idea that some might be susceptible to recruitment by bad actors. This piece references some research and warns security groups to be wary.
There is no shortage of books, or television and movie scripts that might show creative ways to access information, but how can you tell if a colleague makes a simple mistake or they are a bad actor? Clicking on a phishing email could be either one. Not removing anonymous access to an S3 bucket could be either. Losing their credentials through social engineering is something that happens every day. Who’s to say that this happened purposefully?
I don’t want to second guess the people I work with making mistakes, but I also think these possibilities are why we want to use our computer systems with strong auditing and multiple groups reviewing logs. We might not necessarily stop all activity, but we can often detect it quickly and mitigate the issues. It’s also why DevOps and automated deployments with logging are a good idea. They can limit the problems from both accidents and malicious actors.
My employer has started to do more education around security and how individuals can avoid accidentally causing issues. We use a lot of automation, and more all the time, that ensures once we know how we ought to patch and update systems, we can do it regularly and confidently. Repeatable, reliable deployments of changes are what we aim for.
We know they’ll be some mistakes, but we also know that we can quickly identify issues (MTTD) and fix them (MTTR). Even if we get a bad patch from a vendor, we can quickly deploy a “fix” if we get one, or even reinstall and re-patch to lower levels, if needed.
DevOps, GitOps, and other xxOps aren’t just about getting new features out quickly. They also include the ability to fix problems when the need arises. They don’t prevent rogue actors from causing issues, but they should help you detect and recover quicker than you might expect.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I’ve known Adam and Patrick for a long time and I’m thrilled with the content they produce on their Guy in a Cube show. It’s a great resource for anything Power BI, and now, starting to cover Microsoft Fabric. I’ve … Continue reading →
I was lucky enough to attend the very first PASS Summit in 1999. It was a brand new event, and while not overly large, it was busy and crowded in the basement of a Chicago Loop hotel. I had the chance to meet Kalen Delaney there and ask her a question. That was the highlight of the trip, but I also learned a lot about SQL Server 6.5 and 7 there, spending all the time I could in sessions and hanging out after each one listening to the questions others asked the speakers.
Since then, I have been lucky enough to go to most of the Summits. The event has changed a bit, and I look forward to going to see friends each year and re-connect with them. I’ve also loved meeting new people, often those I’ve corresponded with online. Each event has been memorable and exhausting at the same time. It’s a busy week, and I can sometimes feel overwhelmed. However, it’s always felt like it was worth the trip.
I saw a video from Kendra Little where she talks about what she’s looking forward to this year. She has a few things on her mind: connecting with people, learning from the people who write the software, learning from those that use the software, broadening her horizons, and being a part of the data community.
My thoughts are similar. Last year I was looking forward to the event in person, but apprehensive with all the commitments I had for Redgate with speaking. This year I’m much less involved, working more on things before the Summit and I’m hoping I can enjoy the experience more with less work during the event.
For many years I was always excited about the SQL Server Central party. That was a highlight for me and I was sad when the referrals that funded it went away. Perhaps I’ll get find a way to get it back in the future. For now, I want to connect with more people. I want to do this casually in a few ways. From the #sqltrain on Sunday to a few quiet dinners with friends during the week to taking time at night to attend some of the events that various vendors will sponsor, that will be a great chance to bond with friends (new and old) in a social way.
I also want to connect more professionally with people, which I’ll do in the Community Zone, as Kendra suggests, but also around the convention center. I am planning on not having to rush to many things, which means I’ll have more time in hallways and after sessions to talk tech with speakers and attendees. I’m always amazed by the ways some of you use databases and software, and I find myself learning creative solutions that I might suggest to others in the future.
The last thing that I’m looking forward to is the chance to motivate a few more people to run SQL Saturday events in 2024. We’ve had some new events in 2023, a few more than 2022, but not nearly as many as we used to see each year. I’m hoping to connect with more community leaders and volunteers to try and get them to consider organizing an event in the future.
The Summit is a great investment in your career as a data professional, and it can be a good investment for your employer. Make a case, show them you want to learn and grow, and that what you learn there will help you in your job. It’s a park for those of you that bring value to your employer and the can help with retention. You’ll gain knowledge and make contacts that help you on a daily basis.
I hope to see you in Seattle this November.
Note: The price goes up Sept 20, so get registered before then.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
One nice thing with Flyway Enterprise is that it will automatically generate the undo scripts for migration scripts. However, it used to be that finding these and seeing the script was hard. It’s easier after Flyway Desktop 6.5.2
I’ve been working with Flyway Desktop for work more and more as we transition from older SSMS plugins to the standalone tool. This series looks at some tips I’ve gotten along the way.
Undo ScriptsIf you have Flyway Enterprise, you can have Flyway Desktop automatically generate scripts: both forward (versioned) and undo scripts.
The config option for this is in project options, which is in the upper right.
Clicking this
Once have this, if you add a new object, when you generate the migration script, Flyway Desktop will generate the undo script. In the older UI (pre 6.5) this was a script below the generated script, which meant you had to scroll down, or up and down to compare the forward and undo scripts.
The new UI is better. Now there is just a button. Let me show you how to do this.
First, I have a few changes in the schema model. I’ll select one.
Now I’ll generate the scripts. When I do that, I see the two scripts. The versioned (forward) script is first, and you can see the “Undo” at the bottom.
Below this, the rest of the undo script is visible if I scroll down.
It’s a bit of a pain to scroll up and down, so there is a small improvement in the migrations tab. If I go there, I can find my script at the bottom of the list and click it.
At the top of the script, in the bar, there is now a set of toggle buttons for version and undo.
I can click back and forth to easily see both scripts. The V script is above, and the undo is below.
Try it out today. If you haven’t worked with Flyway Desktop, download it today. There is a free version that organizes migrations and paid versions with many more features.
Video WalkthroughI made a quick video showing this as well. You can watch it below, or check out all the Flyway videos I’ve added:
merrenness – n. the lulling isolation of driving late at night – floating through the void in an otherworldly hum, trailing red jewels in the darkness, your high beams seeming back and forth like a lighthouse.
This is certainly something I feel often. I don’t go out late a lot, but there are times I’m coming home and I feel merrenness. Not so much the side to side of high beams, but the up and down on the hilly country roads in Colorado. A bit of side to side as I get near the house.
This is especially something I feel in the Tesla, where there can be a hum and not much else. Watching the world go by, sometimes with music as a background, sometimes silence, reminds me of the wider world around me, both there and muted by the darkness.
From the Dictionary of Obscure Sorrows
I saw this article on using AI to help that included a few prompts. I decided to try one.
This is part of a series of experiments with the ChatGPT and other AI systems. Lots of Copilot lately.
The PromptHere’s what I wrote, since I was reading piece on cloud migration and writing an editorial:
I am currently learning about cloud migration. Convert the key lessons from this topic into engaging stories and metaphors to aid my memorization
The Response1. The Cloud Island Adventure:
Imagine your business as a group of settlers on a remote island. They’ve been working with outdated tools and limited resources, making progress slow and inefficient. One day, they hear about a magical cloud island that’s full of modern technology, abundant resources, and a vibrant community.
The settlers decide to embark on a journey to this cloud island, symbolizing your migration to the cloud. As they arrive, they realize they can instantly access powerful tools, scalable resources, and connect with experts from various fields. Just like in cloud migration, your business gains flexibility and efficiency, leaving behind the struggles of managing on-premises infrastructure.
Key Lesson: Cloud migration is like moving to a new land of opportunity, where you can harness advanced tools and resources to grow your business.
2. The Cloud Highway Construction:
Think of your business as a growing city with a bustling population. However, the existing roads and bridges are becoming congested and outdated, hindering growth and causing delays. To expand and flourish, the city planners decide to build a futuristic cloud highway.
This cloud highway represents your migration to the cloud. Construction crews work tirelessly to build this highway, which is scalable, adaptable, and has lanes for various services. As businesses start using the highway, traffic flows smoothly, and they can easily scale up or down as needed. The old congested roads symbolize the limitations of on-premises systems, while the cloud highway signifies the ease of scalability and innovation in the cloud.
Key Lesson: Cloud migration is like building a high-speed highway that allows your business to easily adapt, grow, and efficiently navigate the digital landscape.
3. The Cloud Carnival:
Envision your business as a traditional carnival that’s been running for years. The carnival’s attractions are fixed in place, and changing or upgrading them is difficult and time-consuming. Visitors are getting bored, and the carnival is losing its charm.
Deciding to modernize, the carnival organizers pack up and move to a new location, which represents migrating to the cloud. In this new location, they can easily set up and tear down attractions, experiment with new games, and respond quickly to changing trends. The cloud becomes their dynamic playground, allowing them to create delightful experiences for visitors.
Key Lesson: Cloud migration is like relocating a carnival to a space where you can easily innovate and provide fresh, exciting experiences to your customers.
4. The Cloud Puzzle Factory:
Imagine your business as a puzzle factory, creating intricate puzzles with limited resources and machinery. You’re tasked with making bigger and more complex puzzles, but the factory’s machinery is old and cannot keep up with the demand.
To overcome this challenge, you decide to move the factory to a cloud-powered puzzle factory. Here, you have access to state-of-the-art machines that can quickly produce even the most complex puzzles. You can rent additional machines when needed and return them when the demand decreases. The cloud puzzle factory allows you to efficiently meet demand without being limited by physical constraints.
Key Lesson: Cloud migration is like upgrading your puzzle factory to a cloud-powered version, where you can easily scale resources to meet demand and improve production efficiency.
By associating these engaging stories and metaphors with the key lessons of cloud migration, you’ll find it easier to remember and apply these concepts in your learning journey.
ThoughtsI liked these analogies. They got me thinking about how to explain things to customers, though they are all positive and don’t represent the downsides. However, when I asked to alter this to show how this hinders progress, I got the result below.
Very interesting. Not sure if I think this is that helpful, but it was thought provoking.
1. The Stagnant Island Settlement:
Picture a group of settlers who refuse to leave their old, resource-scarce island. They struggle with outdated tools and limited resources, but they’re unwilling to explore the nearby cloud island that offers modern technology and abundant resources.
As time goes on, the settlers fall behind in their development. They can’t innovate or expand due to their lack of access to advanced tools and resources. The once-promising settlement becomes stagnant and isolated, symbolizing the limitations of staying on-premises instead of migrating to the cloud.
Key Lesson: Choosing not to migrate to the cloud can lead to stagnation and missed opportunities, as your business remains bound by outdated tools and restricted growth.
2. The Congested Road City:
Imagine a bustling city that stubbornly refuses to build a modern cloud highway. The existing roads and bridges are constantly congested, causing delays, frustration, and hampering the city’s growth.
As more businesses and people move in, the traffic situation worsens. Without the scalability and adaptability of a cloud highway, the city’s infrastructure can’t handle the increased demand. This congestion and lack of flexibility mirror the challenges of sticking with traditional on-premises systems.
Key Lesson: Failing to migrate to the cloud can result in congestion and inefficiency, preventing your business from scaling and responding effectively to changing demands.
3. The Stagnant Carnival Grounds:
Consider a carnival that’s stuck in its old location, unable to adapt to changing times. The attractions become outdated, and visitors start losing interest. Despite the need for innovation, the organizers resist moving to a cloud-enabled carnival.
As a result, the carnival fails to attract new visitors and struggles to create exciting experiences. The reluctance to embrace the cloud leads to a decline in customer satisfaction and revenue, as the competition adopts modern approaches.
Key Lesson: Not migrating to the cloud can result in a decline in customer engagement and revenue, as your business fails to adapt to changing market demands and deliver innovative experiences.
4. The Inefficient Puzzle Factory:
Visualize a puzzle factory that clings to its old machines and refuses to transition to a cloud-powered system. The factory can’t keep up with the demand for larger and more complex puzzles, leading to delays and frustrated customers.
Without the ability to scale resources on demand, the factory struggles to meet production goals. The reluctance to modernize impedes efficiency and profitability, as competitors who utilize cloud resources outpace them.
Key Lesson: Neglecting cloud migration can lead to production inefficiencies and missed business opportunities, as your organization struggles to scale and respond to market needs.
By exploring these scenarios that depict the hindrances of not moving to the cloud, you can gain a better understanding of the potential pitfalls and disadvantages of staying with traditional, on-premises systems.
A customer had a question about restoring with standby, so I wrote a quick post to explain how this works. Another post for me that is simple and hopefully serves as an example for people trying to get blogging as … Continue reading →
This was a provocative title: 6 ITOps Skills That Will Never Be Automated. In a time when AI use is growing quickly and many people fear for their jobs, it’s nice to see someone writing about areas that AI will … Continue reading →
I’m heading to Chicago today for the Redgate Database DevOps in a Day workshops. This is the first of many on the US tour. I’ll be at 5 of them, but there will be another 5 where Ryan, Grant, or … Continue reading →
One of the things that I feel is important to building better software is testing your code. It should be easy and simple to test code and determine if a chance or enhancement has broken anything. This usually requires a … Continue reading →
When Power BI was released, I was working with customers to try and adopt DevOps for their database code. At that time, I thought the Microsoft developers had made an extremely poor decision with the PBIX format in that it … Continue reading →
Working with various Flyway configuration options used to be a pain since they were either CLI parameters or in a text files. We’ve made editing these easier in Flyway 6.5.4. I’ve been working with Flyway Desktop for work more and … Continue reading →
I was checking some arguments in the RESTORE command for SQL Server and saw that the MEDIAPASSWORD option was deprecated and marked as being removed at some point. That made sense, and I assumed that PASSWORD was the option to … Continue reading →
mahpiohanzia – n. the frustration of being unable to fly, unable to stretch out your arms and vault into the air, having finally shrugged off the burden of your own weight, which you’ve been carrying your entire life without a … Continue reading →
I’m heading to Houston today for the Redgate Database DevOps in a Day workshops. This is the first of many on the US tour. I’ll be at 5 of them, but there will be another 5 where Ryan, Grant, or … Continue reading →
I sent some code to a customer recently to help them decrypt some stored procedures. I sent a quick and dirty set of code, noting at the bottom that the results were in XML and needed to be extracted. The … Continue reading →
Earlier this year I was watching someone present on DevOps and IaaC (Infrastructure as Code). The speaker was showing how they had worked with clients to implement tests and checks that evaluated whether their systems were deployed and the code … Continue reading →
I had a client that was struggling with some encrypted stored procedures. They needed to decrypt them, which I know is a pain in the #@$%@#$@#$#@. I had to do this one. This post shows how I sent them some … Continue reading →
A group of people are protesting more cars in San Francisco by disabling self-driving cars with a cone on their hood. It is likely that the cone disables some sensors and won’t let the cars move. For truly autonomous cars, … Continue reading →
The hot new technology of the year is AI. Between ChatGPT, Copilot, and generative AI, it seems that this is invading the world of computing at an incredible rate. Whether this becomes really useful and valuable or not is something … Continue reading →
I’m back from an office trip this week to Pasadena. I had the chance to go to the office, do some internal training for people, and spend time with our US reps and SEs. It’s valuable for me to get … Continue reading →
It’s a small thing, but copying the migration number can be a pain. However, we’ve made this easier in Flyway 6.5.4. I’ve been working with Flyway Desktop for work more and more as we transition from older SSMS plugins to … Continue reading →
I’ve been very lucky in life to go to many conferences throughout my career. I’ve gone as both an attendee and a speaker, and I have found them to be valuable in helping me continue to grow and thrive in … Continue reading →
rückkehrunruhe – n. the feeling of returning from an immersive trip only to notice if fading rapidly from your awareness, as if your brain had automatically assumed it was all just a dream and already went to work scrubbing it … Continue reading →
As a part of a recent Data Exposed that I was on, there was an ADS update which mentioned Copilot being added. Since I’ve been experimented, I decided to give this a try. This is part of a series of … Continue reading →
Inside Redgate Software, someone posted a picture and was asking if anyone knew who the person was. In this case, there had been a conversation at an event, and a picture was taken, but our employee couldn’t remember the individual’s … Continue reading →
Today is a Redgate event in Pasadena and I know the day will get away from me. Since I can’t manage the site or respond, you get a republish of Code Building Code.
If you’re like me, you sometimes wonder how different other environments are from the one I work in. Well, the ones I used to work in. These days I see lots of customers environment, build PoCs, and help them solve … Continue reading →
I had a client that was struggling with some encrypted stored procedures. They needed to decrypt them, which I know is a pain in the #@$%@#$@#$#@. I had to do this one. This post shows how I sent them some … Continue reading →
A long time ago a software engineer advised me to try and ensure that I made my interfaces clear to users, especially those that are busy focusing on some other task the software enables. The phrase he used was to … Continue reading →
When a Flyway Desktop (FWD) project (or Flyway project) has been around for a long time, there can be a lot of migration scripts. That can be a pain for users, but there is a way to find your changes … Continue reading →
I was chatting with someone recently that’s younger than me. They’ve got a good job and had some success learning, but they felt like their career wasn’t progressing fast enough. This person still wanted to grow their career and their … Continue reading →
aubadoir – n. the outworldly atmosphere just before 5 am, when the bleary melodrama of an extremely late night becomes awkwardly conflated with the industrious flourescence of a very early morning. I haven’t seem 5am from the previous day side … Continue reading →
On Aug 22, 2023, I’m co-hosting a webinar with Anderson Rangel, Redgate Solution Engineer in Brisbane. You can register here for the 11am AEST webinar. No, I’m not going to Brisbane. I’d love to, but I’ll be in Colorado and … Continue reading →
Years ago Redgate did some traveling events under the SQL in the City brand. These were a lot of fun and kind of amazing. One of the longer tours also made me realize I would hate being in a musical … Continue reading →
I chose the title slightly to poke at Stack Overflow (SO), but the same take expressed in this tweet could be said about SQL Server Central. It’s not quite the same as anyone can answer questions on SQL Server Central. … Continue reading →
I had a client ask about how to deal with encrypted stored procedures in their database. This post looks at how to find them and I’ll have future posts that show how to decrypt these and also how Flyway helps. … Continue reading →
This T-SQL Tuesday is from a new host, Josephine Bush, leader of the Boulder group just North of me. It’s an interesting invitation, asking what our job titles really mean. I like this as the titles do affect how our … Continue reading →
A client asked for a summary of changes, so I wrote a post to show where to find this in SQL Compare 15. As I was checking out the Summary, I realized there are different ways to present this info, … Continue reading →
Microsoft Fabric was announced at Build in May 2023. This is the next evolution of data warehousing from Microsoft, folding in Synapse and a number of other technologies to create a simpler location for storing and analyzing data. We’ve published … Continue reading →
The start of learning at the 2023 PASS Data Community Summit is 100 days away. I checked. The Summit starts for many of us on Monday with precons or an early arrival and spending time with friends. However, the first … Continue reading →
It’s been a little while since I’ve had time to relax a bit and try some AI help. This is another experiment I made. A user on SSC asked about PowerShell to copy files with a date appended. This is … Continue reading →
The great post-pandemic, post-Great-Resignation, hybrid/remote/in-the-office work debate continues. It seems almost every week I see more stories that report on, hype, attempt to prove, or otherwise stoke emotions about whether the future of work for many people is more likely … Continue reading →
idlewild – adj. feeling grateful to be stranded in a place where you can’t do much of anything, which temporarily alleviates the burden of being able to do anything at any time and frees up your brain to do whatever … Continue reading →
A friend asked me for some travel tips and ideas recently. One of their questions was on what types of things do I actually carry on the road, both for work and personal trips. Since I travel a lot, 18 … Continue reading →
I had to find a set of identity columns recently and through this would make a good blog post. Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. … Continue reading →
I will admit that I don’t know a lot about AI (Artificial Intelligence) systems and how they are built. I’ve been playing with them a bit and haven’t been overly impressed with the results. I think some of this is … Continue reading →
A client asked for a summary of changes, so I wrote this quick post to show where to find this in SQL Compare 15. If you use SQL Compare, you might find yourself in situations like this, where there are … Continue reading →
At Redgate Software, we’ve been trialing Copilot from GitHub with our developers. I managed to get access for this experiment and have tried a few things, though I’m not sure I’ve found it very useful. I’ll continue to work with … Continue reading →
I heard a joke years ago that went something like this. When a developer gets a pull request for code review that’s 100 lines long, they will open the file(s), look at the code, and ensure standards are being followed. … Continue reading →
fitzcaraldo – n. a random image that becomes lodged deep in your brain – maybe washed there by a dream, or smuggled inside a book, or planted during a casual conversation – which then grows into a wild and impractical … Continue reading →
I had to test something for a customer, and as a part of this there as a need to have a different default schema for a user. I wrote about that, but one of the things that occurred to me … Continue reading →
I guess I’m technically working today. I’m at That Conference in Wisconsin, speaking and networking with fellow software geeks, but I’m expecting to be mostly out of touch, I’m republishing The Minimum Upgrade Point
Actually, this is a change for many products and software that connects to SQL Server. Updated drivers require us to now decide to trust the server certificate. I opened a project recently in SQL Compare 15 to check something for … Continue reading →
The early bird pricing for the PASS Data Community Summit ends this week, on Jul 26. After that, there is a bump, so let you boss know this is the time to register. If you’re considering attending, then you essentially … Continue reading →
Every quarter Brent Ozar publishes some data from his SQL ConstantCare® service. This is a service where companies contract with Brent to install a service on their instance, collect data, and give them simple, short daily emails on things they … Continue reading →
I was working with a customer recently that has a development process that both made me cringe and struck me as very creative. In this case, the customer has software they have written when spawning a few databases for each … Continue reading →
Ozurie – feeling torn between the life you want and the life you have. I think many people feel ozurie often. I certainly had a lot of this in my younger years. I’d see the success, the partners, the adventures, … Continue reading →
I had to test something for a customer, and as a part of this there as a need to have a different default schema for a user. I wrote about that, but since this isn’t something that I (or many … Continue reading →
Most of the code I’ve written for various employers was something that lived inside the organization. This was for internal use and no one outside of the organization ever used it. It was almost always code I’d written, so I … Continue reading →
I’m a little buried. Life has been crazy and I am out of town again. Actually traveling today, so I’m republishing Abolish Disjointed Time. I am still not happy with DST this past spring.
exulansis – n. the tendency to give up trying to talk about an experience because people are unable to relate to it – whether through envy or pity or mere foreignness – which allows it to drift away from the … Continue reading →
One trend for many organizations is to be data-driven. This means using data to make decisions at all levels, or at least support those decisions. This was popularized by many companies, and there is research to back up the claims … Continue reading →
Earlier this week the emails went out to speakers who submitted to the PASS Data Community Summit 2023 conference. These were acceptances and rejections, letting people know the results of the volunteer review. I got this one: I had only … Continue reading →
I live on a working horse ranch. My wife boards, trains, and trims horses and has employees. Occasionally I have to help out with chores, or more often, fixing things. I was talking with a coworker recently and showing some … Continue reading →
I had to test something for a customer, and as a part of this there as a need to have a different default schema for a user. Since this isn’t something that I (or many people) do often, I wanted … Continue reading →
The invitation this month is from Erik Darling, and it’s a neat one. I like this thought, asking us to find code that impressed us or made us feel something. I tend to look at this as positive, but it … Continue reading →
At the first job I had as a DBA, I had to build a new server. This was in the days of SQL Server 4.2, and I was combination DBA, sysadmin, and general help desk at a small company. With … Continue reading →
I give a few talks on career topics, and one of these is Branding Yourself for a Dream Job. In the talk, I sometimes tell a story in the volunteering section. I wanted to summarize that here, as a way … Continue reading →
I was on the Data Exposed: MVP Edition show recently, talking about SQLCMD. I’ve written a few articles on the topics as well, and a blog post about setting up a node HTTP server, which I show in the demo. … Continue reading →
I’m very lucky in that I get to travel all over the globe for work and see many places. It can be hard, and this year has been a challenge. Through the first six months of the year, I’ve taken … Continue reading →
Are you ready to get a new job? I’m not talking about you deciding you want a new job and working towards that goal, but rather getting surprised by a layoff or termination today. If your boss called you into … Continue reading →
licotic – adj. anxiously excited to introduce a friend to something you think is amazing – a classic album, a favorite restaurant, a TV show they’re lucky enough to watch for the first time – which prompts you to continually … Continue reading →
I’ve had to type a few non-English characters lately, and this blog talks about how to do this. Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. The … Continue reading →
I read an interesting blog from the cyber security team at Microsoft, noting you should use TLS for your SQL Server connections. I would assume most professionals know that using TLS and secure protocols across the network is important. I … Continue reading →
This is documented, somewhat, but I wanted to put this down for myself, as the I don’t love the docs and they are hard to sort through. Flyway is open source software owned and maintained by Redgate, my employer. There … Continue reading →
I saw a great question on Twitter from Frank Pachot, a developer advocate of Yugabyte. He wrote: Without thinking how your preferred database deals with it, what do you expect if: session 1 starts to reads table T session 2 … Continue reading →
I don’t know that I have the best advice, but this month’s T-SQL Tuesday is asking for people to share what they think is the best advice they’ve been given or have for you. I wrote my own piece, where … Continue reading →
volander: n. the ethereal feeling of looking down at the world through an airplane window, able to catch a glimpse of far-flung places you’d never see in person, free to let your mind wander, trying to imagine what they must … Continue reading →
This week I had the chance to deliver a talk at Agile West in Las Vegas. I linked some resources on my blog, and feel free to check them out. Tl;Dr – Start with deployments After the talk, I had … Continue reading →
I’m in the air again, hopefully. I should have taken off about 20 minutes ago for Orlando. This time, however, it’s not work, it’s personal. This is a short break, heading off to the AAU Girls Junior National Championships. My … Continue reading →
We’re a little delayed this month. Both the host and I forgot about this. So far in June, I’ve been in Fort Lauderdale, Las Vegas, Denver, and Cambridge for events. I head back to Orlando this week, and I’m a … Continue reading →
With some projector issues at SQL Saturday South Florida 2023, I had to lecture without demos this past weekend. I told everyone I would record the demos, so I’ve done that. The three demos are shown below, and if you … Continue reading →
The world of work has been changing quite a bit over the last few years. The pandemic appeared to be a boon to technology workers, both by allowing us to work remotely, but also taking advantage of our skills to … Continue reading →
I’ve been working with Ryan Booz a bit more and as we’ve talked over the last few weeks, he has asked me a few times if I’ve booked travel for That Conference, or if I’ve gotten my presentation ready for … Continue reading →
astrophe n the feeling of being stuck on Earth. Us geeks are supposed to love space, right? Science Fiction? The Moon is a Harsh Mistress? Star Trek/Star Wars/Lost in Space? The desire to travel throughout the universe? I know I … Continue reading →
I’m in Cambridge, UK this week for the internal Global Marketing Week and Level Up Conference. This is my first visit to Cambridge in 2023, and we have a new office. The old one is literally next door, but after … Continue reading →
SQL Server databases have had a compatibility level for a long time. This is a setting that enables the database to process code as if it were a particular version. The levels go from 80 (SQL Server 2000) to 160 … Continue reading →
I’m in the UK again, for my second trip this year. This time I have no commitments for speaking or presenting anything. I’m in town for a Marketing get-together and our internal Level Up conference. Nice to be here without … Continue reading →
For those that attended my talk at Denver Dev Days, here are the slides: BloggingFortheTechPro.pptx A couple interesting questions that I need to add to the deck. What do you recommend for a student? (or someone early in their career) … Continue reading →
As a part of my AI experiments, I decided to ask CoPilot to write some unit tests. Here is what happened. The Prompt To get started, you enter a prompt as a comment and put the cursor on the next … Continue reading →
There are a lot of technology people looking for jobs these days, especially after all the layoffs that have occurred in 2023. At the same time, I have a number of friends and clients that are struggling to hire qualified … Continue reading →
slipfast adj. longing to disappear completely; to melt into a crowd and become invisible, so you can take in the world without having to take part in it – free to wander through conversations without ever leaving footprints, free to … Continue reading →
I delivered a talk this past week at Agile West called “Don’t Forget the Database” Slides posted here: DontForgetTheDatabase.pptx This is based on code and the app in my ZeroDowntime repo.
Redgate is sponsoring Agile West this year, and the marketing team asked me to give a talk on database DevOps technologies.
I have a keynote talk, called Don’t Forget the Database, that I’ll be delivering on Wednesday. This covers some of the challenges of why the database is harder than software, but doesn’t need to be. I show some demos of zero downtime deployments as a part of this talk.
Likely I’ll record the demo and talk over it, as I have seen that technique work well and it keeps me from fumbling around.
My wife is coming with me. This is a quick Tues-Thur trip for me, and easy. Las Vegas is about an hour flight for me, and I only have a few commitments, so this is a chance for the two of us to see a show or two and get out of town for a short break.
I’m lucky that some of my business travel allows my wife to come around, and it’s worth the expense to bring her and make the trip more enjoyable. That makes it easy for me to handle the 20-30 trips a year I make.
If you’re at the show, stop by the Redgate booth or my talk and say hi.
I saw a great post from DCAC on disaster plans and using them after a fire in an LA data center. DCAC wasn’t affected, and I wouldn’t expect them to be. Denny, Joey, Monica, John, Kerry, and Meagan are all experts in running systems efficiently and effectively. They think about how to ensure that things keep working in the event of disasters and have delivered presentations all over the world helping you become better at managing your own servers.
However, Denny brings up a great question in the post. How many days could you survive without IT systems? Days? Weeks? Have you ever been in that situation? I’m sure many of us have experienced a failure of some sort. An application crash, hardware dying, or most commonly, Internet access being cut. All of these are small disasters, which typically are fixed quickly. Hopefully, these aren’t issues you experience every week. If you do, maybe you need to call DCAC or someone else and fix more fundamental issues.
While it’s unlikely that you lose all your systems, it could happen if you concentrate your resources in one data center, one region, one cloud provider, etc. Having a single point of failure is something we try to avoid in IT, and that is true not just for one application, but for your infrastructure design. Most of us depend on one authentication system, and a failure of our Active Directory could lock everyone out of a system. Those are rare, and hopefully, your administrators have enough redundancy (and backups) to recover from this type of disaster.
I have experienced a few large failures at one enterprise. We had a few viruses, including the SQL Slammer worm in the early 2000s. Our network was shut down for a couple of days, when almost all systems from email to CRM to ticketing systems weren’t available. Everyone had to use whatever paper systems they could to keep business running. While we likely lost some revenue from these outages, we learned we could survive a few days without our network. We also learned that we needed better virus scanning and education for employees, as well as a few more resources for tech people. Before those events, everyone assumed an outage was bad, but had no idea how bad. I have no idea how much this cost us, but it didn’t appear in a 10-Q, so it must not have been too bad.
I think there are lots of businesses that could find ways to continue to work if some systems were down. However, there can be costs, sometimes significant. Even if the company doesn’t go out of business, perhaps some people get terminated because of less revenue. That might not be the tech person in the short term, but how would you feel if your DR plan didn’t work (or you didn’t have one) and some co-workers were let go?
The move to the cloud, and the move to more software-as-a-service systems, might help you better survive local disasters, but if you have too many systems concentrated in one place, it is worth preparing for some contingency. After all, even if this fire were to happen in an Azure or AWS data center, it’s possible that their process to move and restore all the systems from one data center to another could take time. Your systems might not even be their top priority as cloud vendors have some large customers. It probably won’t take months, but I wouldn’t want to bet my job on any cloud vendor getting everything moved in less than a week.
If you’re not in the cloud, make sure you have a plan. If you don’t know how to do that, call DCAC or another consultant to help you.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I get asked questions by salespeople all the time. Usually this is because a customer is struggling and I try to educate the salesperson as well as give them something to send to the customer. Here’s a place I used ChatGPT to make this quicker, or at least try.
This is part of a series of experiments with the ChatGPT and other AI systems.
Asking a QuestionA customer asked about configuring SQL Backup Pro to use Amazon S3 for backup locations. Rather than search out the docs, find URLs, etc. I asked ChatGPT. Here’s what I got:
Prompt: How do I configure SQL Backup Pro to send files to Amazon s3?
Answer:
To configure SQL Backup Pro to send backup files to Amazon S3, you need to set up an Amazon S3 storage account and configure SQL Backup Pro to use it as the destination for backup files. Here are the general steps to configure this integration:
- Create an Amazon S3 bucket: Log in to your Amazon Web Services (AWS) account and create an S3 bucket where you want to store the SQL Server backup files. Take note of the bucket name and the AWS access credentials.
- Install SQL Backup Pro: If you haven’t already, download and install SQL Backup Pro from Redgate Software. Ensure that it is properly installed and licensed.
- Open SQL Backup Pro: Launch SQL Backup Pro and open the SQL Server instance you want to configure for backups.
- Configure backup settings: In SQL Backup Pro, navigate to the Backup tab and configure the backup settings according to your requirements, such as database selection, compression level, and encryption settings.
- Configure backup destination: In the backup settings, choose the backup destination as “File (network or local drive)”.
- Set the backup path: Specify a local folder where SQL Backup Pro will temporarily store the backup files before transferring them to Amazon S3. This can be any local folder accessible by the SQL Server service account.
- Configure transfer settings: Under the “Backup files to the network or local folder” section, select the option “After the backup completes, copy files to another location or device”.
- Choose the transfer type: In the transfer settings, select the transfer type as “Amazon S3”.
- Enter Amazon S3 credentials: Provide the AWS access credentials (Access Key ID and Secret Access Key) associated with your Amazon S3 bucket. These credentials authenticate SQL Backup Pro to access and upload files to your S3 bucket.
- Specify the S3 bucket details: Enter the bucket name and the desired S3 region where the backup files should be uploaded.
- Set other transfer options: Configure additional transfer options, such as file naming, retention policies, and logging preferences.
- Save and test the configuration: Save the settings and perform a test backup to verify that SQL Backup Pro can successfully transfer the backup files to Amazon S3.
Once the configuration is complete, SQL Backup Pro will automatically transfer the backup files to the specified Amazon S3 bucket after each backup operation.
Note: The specific steps and options may vary slightly depending on the version of SQL Backup Pro you are using. It’s recommended to consult the product documentation or contact Redgate support for detailed instructions based on your specific version and environment.
Did it Work?While this was generating, I actually looked for docs to evaluate whether this was good advice. I don’t completely trust the AIs yet. What I found was:
- The doc here: https://documentation.red-gate.com/sbu/backing-up/creating-backups/creating-backups-file-settings, but no good link to the Hosted Storage section, which talks about what to do.
- A separate answer from someone else pointing me here: https://documentation.red-gate.com/sbu/worked-examples/backing-up-to-cloud-storage-settings
- In the walkthrough above, you really need to link your account, which is done from here: https://documentation.red-gate.com/sbu/worked-examples/backing-up-to-cloud-storage-settings
The instructions from ChatGPT don’t reference specific URLs, which I think is something that I would hope for. If I were sending instructions to a client, they might figure things out from the ChatGPT answer, but they’d be annoyed.
A good example of where domain knowledge is needed, and still some work. This might be helpful if I had links it the answer to quickly check things.
jouska – a hypothetical conversation that you compulsively play out in your head – a crisp analysis, a devastating comeback, a cathartic heart-to-heart – which serves as a kind of phycological batting cage that feels far more satisfying than the small-ball strategies of everyday life.
This happens to me all the time. Maybe this is something I should work on in therapy? However, since I try not to should-on-myself, I’ll see if I can reduce this internal dialogue.
Sometimes it’s productive for me, such as the times I’m gaming what questions someone might ask me about a presentation or how I will pitch something in a meeting. In these cases, running through the “ideal” conversation in my head is good initial prep.
However. I sometimes have this crazy hypothetical conversation in my head when someone has upset or offended me. I’m sure “I’m right”, and I work through all the great things I would say and how I’d show the other person they are wrong.
Things never work out that way, and I hope I find ways to engage in less jouska.
From the Dictionary of Obscure Sorrows
I have learned to really appreciate and enjoy Slack as a messaging tool. It’s something I use daily, and a place where many inside my company communicate about all sorts of issues. There certainly can be an overload of channels, but for me, I add and prune channels regularly and it’s a good way to segregate conversations.
I didn’t feel that way when I started. At first, I resisted using it. Now I couldn’t imagine not having it, but not everyone feels the same way. There’s an article about a SaaS provider moving their company off Slack and instead using the Basecamp project management platform and keeping communications inside there.
Why? They say they have less meetings, less interruptions, less direct messaging, and more productivity.
Interesting conclusions. I certainly can see that some people might find Slack to be chatty, and there are definitely lots of channels devoted to non-work items. There can be a lot of unread channels from me, but I don’t know that we would have less meetings without Slack. We have various work tracking and productivity tools, like SalesForce, trello boards, etc. However, those aren’t the places I find it convenient to move communications.
Personally, I don’t have a lot of DMs, and I like that I can see unread channels, ignoring them when I’m busy and looking at them when I have time. I can also just mark-as-read a channel when I want to declare thread bankruptcy.
I usually don’t find Slack to be too intrusive to my day. I work fairly independently and asynchronously, as I’d expect many developers to work. I also don’t know that it causes more meetings, at least not for me. There is a fairly high bar to opening a meeting and I find many people don’t bother. Instead, we can have a discussion, in real time or across days, that deals with a topic.
What do you think? If you use Slack (or Teams/etc.) is this a drain on productivity or an enhancer? I think the latter.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
The PASS Data Community Summit is back in Seattle this November, the 14-17, 2023. It’s in person only, and if you can come, you should register today. Launch pricing is discounted and ends next week, June 7.
Complete pricing is on the site, but next week the prices rise, not only for the event, but the pre-cons as well. $500 for a pre-con is a great price, and it’s not something you see from many of the experts when they run private training. The conference is also 30% off now, so let your boss know now is the time to register.
You can see the pre-cons listed, and a number of sessions on the schedule as well. There is a lot of learning and more to come. This isn’t a complete list of sessions as the volunteers on the program committee are still making their final decisions.
If you need to make a case for your employer, there is some great information on the site about what you can learn and why it’s worth attending. The networking alone, which let’s you meet and talk with experts, can be worth a lot later. One support call and the time spent could be avoided with a new colleague that you can query.
I hope to see many of you there. I don’t know if I’m speaking, but I am going and helping run the event. Come to Seattle this fall (and SQL Saturday Oregon the weekend before) and grow your career and get excited about being a data professional.
A customer asked about how they could organize their migration scripts in different ways to manage them and worried about it being complex. I decided to test a few things. This post looks at using multiple folders for scripts. Setup … Continue reading →
Monitoring databases is important when it’s the systems that are in production. Operations departments know that catching issues early, being proactive, and having data to troubleshoot issues make their job easier. Not having these things makes their job much more … Continue reading →
Recently I got a message that my Evernote subscription was going up. It’s been a $3 a month service, but moving to $4 for me. From USD$35 to USD$50 a year. Not a big change, but a little annoying to me. I have a few reasons, outlined below, but this post is mainly describing the process of moving my data.
Setting up Joplin is easy. You download the app, install it, and then (optionally) connect it to Dropbox, where it adds itself as an app in the /apps folder.
Export and ImportI can easily export a notebook in Evernote to their ENEX format. I do this by right clicking a notebook and selecting Export notebook.
This asks me how to store the output. I get a single file, pick a name/location and this creates a file.
Once this is done, in the Joplin desktop app, I have a few choices. I pick ENEX and then the type, select the file, and things import.
I tried HTML, but got a mess.
If I could edit in the right pane, this would be OK, but I have to edit in the source pane, which is annoying. I know HTML, but don’t want to write in it.
Instead, I picked markdown, which gives me a better view.
I have perhaps a few thousand notes, but really only about 12 notebooks. Since I can export 12 times and import 12 times, this is easy.
I then have all my notes in a new app, where I can sync them between desktop, mobile, and the iPad in the kitchen.
Note I’ll spend a couple weeks working in Joplin and see what I think.
Why Leave Evernote?Evernote has continued to expand their capabilities, which I get. As a software service, they want to keep growing and attracting more customers and providing more features.
However, for me, I want to simply take notes. I almost never include images, web clips, etc. Instead, I like simple text and quick software. I want to make notes and save them, syncing across devices. To me, I need a glorified way of capturing text files and moving them between machines.
While Evernote has worked well, they’ve done some things that make this less ergonomic for me. Specifically:
- They add a div tag that notes this item was clipboard’d, which is perpetually annoying. I also copy/paste often as this is my writing tool and they add some markup in HTML, which I then have to remove.
- New notes mean I have to select if this is a note, a task, or something. One extra, very, very annoying click.
- Their navigation options are clunky and slow on mobile/tablet.
- Their UX has me constantly clicking from the top to the bottom to the top of the screen.
- They’ve become more chatty, and single-threaded with more lags. Something I find annoying.
They have also deprecated the Plus subscription, which is simple.
I decided to try Joplin, which I found in this PC Mag article. Joplin is an open source app, and if I keep using it, I’ll donate some money to the developer. I don’t mind paying for software, but I need something simple.
If this doesn’t work, I have other options. I might just create a private GH repo and use folders to organize text files. That’s about what I need.
I can also renew Evernote for a year and kick the can and hope they don’t try to force me into the $12/mo subscription.
I know lots of people love OneNote, but I find it overkill and too annoying.
I would guess that the majority of instances I’ve had to manage in my career were those that I didn’t initially install and configure. I’ve inherited more instances than I would bother to count, and I often need to double-check what’s been done in the past. As noted in the series on new jobs from Tracy and Josephine, there are a lot of settings to check and adjust to meet your standards.
While backups are often my first priority, security is second. I usually want to know who the sysadmins are and ensure systems are patched and configured to reduce the attack surface area. There is one other security check that I think I haven’t always been overly concerned about checking: password expiration.
There was a post from Steve Stedman recently that mentioned the way to alter logins and ensure they have CHECK_EXPIRATION set ot on, which ensures that passwords expire and need to get changed. This is especially important for sysadmins. I try to ensure those accounts in that role are secured with AD, but there have been times when SQL accounts are used. Usually, I disable sa, but I’ve seen other accounts, especially those used by monitoring systems who seem to think sysadmin is required. It’s not.
I don’t know that I’ve run queries to check the value in the is_expiration_checked column is appropriately set. If it’s not, then Steve’s post above will help you change those logins. That’s a handy script to have set up and use to ensure that all logins have this set. In fact, this is one of those areas where new logins could be created by junior administrators and not set the option. Perhaps this is something you want to run on a regular basis, perhaps weekly, to ensure that if any new SQL logins are created, they are done so with the password expiration set.
Ideally, no one would ever create logins without expiration set, but sometimes things happen. I’ve seen monitoring systems set up with sysadmin privileges and passwords that never expire. A surefire way to dramatically increase the risk to your database systems. It would be better to have a known, consistent process for setting up accounts. Some companies have specific scripts, or snippets, that administrators use when tickets are filed. One customer of mine had even linked a script to a Slack command in a sysadmin channel. Only admins could use this channel, but they could use Slack to kick off scripts to create logins, add roles, and force password changes.
No matter how you choose to handle security at a process level, it is important to include monitoring and remediation for issues. Mistakes will get made, settings altered, and exceptions approved. Sometimes we can fix things, sometimes we cannot, but knowing what our environment looks like and where we have potential issues is important not only for getting the work complete but getting the approvals to make changes that ensure better security. My recommendation is that you ensure you have a way to regularly check your systems, automatically fix issues where appropriate, and report on those that need additional approvals.
Steve Jones
A customer had a question recently on masking Chinese characters. I thought that was interesting, so decided to test this out. This is a short post on using SQL Data Masker to accomplish this task, but I’ll a longer one on the Redgate Product Learning site.
Setting Up A TableThe first thing here was to get some test data. I was looking for Chinese names, since that was the request. I found this page on the most popular Chinese surnames. With that in mind, I build a small table and a few insert statements with this code. I only used the names Chén, Yáng, Zhào, Huáng, Zhōu, Wú, and one Western name for the demo
CREATE TABLE dbo.CustomerFromChina( customerid INT NOT NULL CONSTRAINT CustomerFromChinaPK PRIMARY KEY , customersurname nvarchar(100))GOINSERT dbo.CustomerFromChina (customerid, customersurname)VALUES (1, N'陈'), (2, N'杨'), (3, N'赵'), (4, N'黄'), (5, N'周'), (6, N'吴'), (7, N'Joe')GO
I ran this and saw the results I needed.
Creating a New Data SetSQL Data Masker ships with a number of masking sets, but you can add your own. There is a process, but essentially you create a text file with the data in it and the udef extension.
The masking sets are in Program Files below the Redgate folder. This is an administative folder, so you need to have admin rights to make a new file. I did that and opened my file in VSCode, which defaults to UTF-8 format. Since I want to use Chinese characters, I need to use a text file that supports unicode.
I entered four names into my test file and saved it. You can see the entries here.
I made the file name, chinesesurnames.udef. Make sure that this doesn’t have the .txt extension at the end.
Setting The Masking SetI opened SQL Data Masker and created a new masking set. I connected this to my database and then went to the Misc. Setup tab. I didn’t see my set (I had this open), so I clicked “Refresh” at the lower left. This brought the data set into the list view. As you can see, I should have capitalized the file name.
.
If I clicked “Sample” at the bottom, I see my data:
Note that the sample window shows a bunch of rows, but they are repeating the same four values.
Next, I added a new Substitution rule. I picked the CustomerFromChina table and the ChineseSurname column. I also selected my custom data set. This is noted on the right of the image below.
I saved this masking set and I was ready to test.
TestingI first connected to the database and ran the query above in one window. Then I opened a vertical tab set, which moved this window to the right.
Next, I ran the masking set, which was quick on 7 rows of data. I then opened a new query window on the left side of SSMS and ran the same query I’d run on the right. You can see the results below:
In the image, the right has the original data. CustomerID 1 is Chén on the right, but on the left, this is now Liú. The others are also changed, with Joe also being masked to Liú.
This worked.
You can do the same thing with custom masking sets, in any Unicode format or language.
SQL Data Masker is a great product for managing a set of update scripts that will remote PII data from a database. It is clunky, but it works well for building and updating a project across time and with a team. Way better than T-SQL scripts.
Give it a try today.
looseleft – the feeling of loss upon finishing a good book, sensing the weight of the back cover locking away the lives of characters you’ve gotten to know so well.
I feel this often. I think this is one reason I read series. Even when they end, this feeling is lessened because there are others to read, or re-read, as I do regularly.
However, Stephen King and Harlen Coben as favorite authors who often write novels about one set of characters that I love and get to know and then forever are stuck remembering the small part of their lives I shared.
From the Dictionary of Obscure Sorrows
I have been experimenting more and more with ChatGPT on a regular basis. I’m interested and curious how this will help. I had a friend at one of the large FANG companies note that they found ChatGPT to be very helpful in their job, with assistance in various things. Their quote:
”…I used it to collaborate, having a design discussion, writing requirements, comments and document my code (!!), improve my writing and find the gap and provide suggestions…”
I thought that was interesting, so I sent in this prompt: write a one page post in the style of http://www.voiceofthedba.com that summarizes this article https://www.madeiradata.com/post/home-experiments-and-stacked-bar-graphs
I often find links that catch my eye, read them, and save them in a note for a future editorial. This time I asked ChatGPT.
This is part of a series of experiments with the ChatGPT and other AI systems.
The ResultsWhat I got from ChatGPT is below. I read it and then wrote my own editorial, which was called The Best Graph. Mine is quite different, and I think, it’s better. I’m not sure the response below summarizes the post correctly, I don’t know it’s in my style (maybe parts), and ultimately, it’s a summary, not a commentary. Maybe that’s on me. I didn’t specify an editorial or opinion.
Read them both and let me know which one you like more, which you think works better.
The ChatGPT ResponseIn a recent blog post on Madeira Data Solutions, author Lisa B. shares her findings on the use of stacked bar graphs in visualizing data from home experiments. While stacked bar graphs are commonly used in business and marketing contexts, Lisa explores their potential usefulness in a personal setting.
Lisa begins by describing her home experiments, which involve testing the efficacy of various cleaning products. She notes that while she initially used line graphs to track the results of her tests, she found that stacked bar graphs were more effective in illustrating the relative performance of each product.
Lisa goes on to provide a detailed explanation of stacked bar graphs and their components, including the use of color-coding and labels to differentiate between different categories and subcategories. She also notes that stacked bar graphs are useful in highlighting changes over time and in comparing data across different groups.
One of the key takeaways from Lisa’s post is the importance of choosing the right type of graph for a given dataset. While line graphs may be appropriate for tracking changes in a single variable over time, stacked bar graphs may be more useful when comparing multiple variables or subcategories. By selecting the appropriate type of graph, researchers can more effectively communicate their findings and insights to others.
Overall, Lisa’s post provides a useful overview of stacked bar graphs and their potential applications in a personal, as well as professional, context. Whether you’re conducting home experiments or analyzing business data, it’s important to choose the right type of graph to effectively convey your findings.
If you haven’t seen the Build 2023 keynote, it’s, well, interesting. At a surface level, it’s focused on AI and delivers some demos that many of us might find to be useful and intriguing. I didn’t attend the event (or watch it live), but I did see it a bit later and I made some notes, pausing the 30-minute talk a few times to think about what I’d seen.
The opening lightly glosses over some of the AI enhancements to development tools and the environments that can be created quickly in GitHub or Azure. Some of us will like those, and maybe they’ll grow on me, but I tend to prefer a development environment on my own hardware, where I have unlimited compute power at a fixed cost. The first big announcement is then showing Copilot technology, essentially some ChatGPT-like abilities, embedded into Windows 11. The demo shows asking Windows where settings are, with the response including buttons to take actions, like setting dark mode. Minorly useful, though I think Windows search works fine. I can type “env” and get the “edit environment variables” in the results. I still have to click through to change things, but this doesn’t seem like a better use of AI, especially if I need to type “set dark mode” instead of “dark”.
To be fair, the demo has the user asking for ways to adjust the system to get more work done. The suggestions are for dark mode and a focus timer. I knew about the former, but not the latter. Perhaps being able to ask for general assistance with tasks is useful as there are likely lots of features I know nothing about and wouldn’t even think to look for. There is also the option to drop a document, like a PDF, in the chat and Windows asks if the user wants the system to “explain”, “rewrite”, or “summarize” the document. The user clicks summarize and gets a summary of the document.
There is also a demo with plugins that developers can write for Bing, such as one that uses a legal package to make a change to a document. While lawyers might be worried about their practices (or paralegals about job prospects), I’m more worried about a fundamental problem that many of us data professionals have seen in the past: garbage in, garbage out.
In this case, if the AI model isn’t well-trained, can I really trust it to summarize a PDF or change a legal document? How can I tell if it’s wrong, or slightly off? In some sense, this reminds me of a high school report. It might summarize some text at an A level, or a D level. It’s up to me to judge that, and I can’t assume the results are good or bad.
The important thing to keep in mind, however, is that we aren’t in that place with AI. We can’t just trust the AI. We are in a time when AI is an assistant, where it can help us complete a task or get something done a little quicker. We are still responsible. We still have to verify and do some work, but if the Copilot can automatically launch Jira and navigate to a ticket, or attach a document and create a short message to our team, that saves us time. It saves us tedium. It can make our jobs easier. We are still needed, but we don’t do all the heavy lifting.
I do worry about some of the opportunities for plugins that developers will write strictly to monetize their efforts. If I want a shopping list, I don’t want it to go to Instacart. I want a list I can use. I realize that doesn’t necessarily make Microsoft or a developer any money, but not all the tasks and advances are about profit. Or at least, I hope they all aren’t. I hope some are here to just make the world better. For a quick view of what that could be, watch the keynote closing video.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I’m going to attend the Denver Dev Days 2023 in June. I submitted a few sessions and got two picked. I’ll be doing these talks:
- Blogging for the Tech Professional
- Architecting Zero Downtime Database Deployments
These are both sessions I enjoy.
If you’re in the Denver area, this is a fun developer learning event at the Microsoft office in the Tech Center. You can get more information here: Denver Dev Days 2023. Hope to see you there.
I delivered a presentation for the Houston SQL Server User Group tonight, virtually from Colorado. I’d love to go down, but couldn’t do it today.
The slides for the talk are here: BrandingDreamJob.pptx
I had fun and thanks to the Houston group and Paresh for having me.
A few interesting questions:
Should you use your real name on Slack (or Twitter/etc.)There is no right or wrong answer here. For me, I want to tie together my efforts, so if I help people on Twitter, I want to link my resume/profile to my Twitter account. It’s why I keep way0utwest and try to get that on new platforms. My profile says “Steve Jones”.
You can do what you want.
What about using ChatGPT to write blogs?A great comment here from Paresh. I think this is a good way to get started and maybe sketch out a post. I would recommend this, though I think this needs to grow and improve.
However.
You don’t know if ChatGPT synthesized sentences and paragraphs from other information, which is what we all do. You also don’t know if it copied paragraphs verbatim from somewhere.
My advice if you use this is plan on editing the result and making it your own.
How do we network virtually?I’ve networked and built friendships online for years. In the SQL Server Central forums and similar places. Sometimes real time chat, sometimes asynchronous posts. However it works.
If you can get to know people with messaging or posts on a platform, that’s great. If you come to a virtual group meeting, or even a company meeting, early, you can spend a few minutes in chat or live chatting with them. Ask a question, share something, solicit an opinion, or comment on something.
In busy chats, or even aloud, you might @ someone or address them. In chat, @Steve, that was an interesting comment. Aloud, “Steve, have you ever used automatic seeing in an AG?”
Setting up a local web server was something that I haven’t done in a long time and this was really easy. This post shows how to do this with a command line node web server.
A NeedMost software that needs a web server sets one up for you. This is pretty easy, and if I have a web app in Visual Studio, it will start up its own server when you run the app. If you had to set your own up, IIS was in many versions of Windows, but it wasn’t a simple thing to configure and manage.
I like simple.
I needed to just serve a file over http to test something. I could have done this by uploading a file to this blog or somewhere else, but I was hoping to keep this small and portable.
Looking AroundAs always, I started with Google. I found a bunch of articles on Apache, python (should have tried this) and others. Then I saw this short article on a test web server on Windows. I looked, and this seemed to be something simple. I had node installed, so really I ran this:
npm install -g http-server
This installed somewhere. I had created a new folder in d:\http, but the web server was installed elsewhere. When I typed “http-server” in the command line, it seemed to be running. I saw this:
I created a quick html file, index.html, with this content:
and voila, I had a web server.
It’s basic, but any file I drop into d:\http can be served over http by entering the file name in the path. You can see in the image above, I was testing moving SQL Server backups. That URL actually downloaded that file from my browser.
SummaryThis showed how you can set up a server with node. It’s quick and easy, and flexible. I can set this up as needed for tests without any big config. If you need this, here’s one way to test your own web stuff.
Conveying information is a bit of an art and science. Many of us have written reports, graphs, charts, etc. at some point in our career. We’ve likely created some good ones and some bad ones that our clients love or hate. Perhaps if you’re like me, you make a small attempt and then ask someone else to clean it up for you.
However, visualizations are important. In the modern world, many people want a visualization instead of a table of data, or at least alongside a table. That means we want to ensure we are conveying information well and not just picking the prettiest picture.
One interesting thing to consider is how different types of graphs affect how we process information. There was a post from Madiera Data that looked at how different graphs helped someone analyze the data. It starts with bar graphs, which have been very popular in the last few years. They are appearing in lots of business dashboards, news articles, and more. They can be useful in some situations, but not all. Especially when trying to compare the different segments.
Instead, other visualizations can be better. The article shows table graphs are working well. I conducted my own experiment with visuals, and I found a line graph was easier to use. However, that was for my data. Your data might need different visuals if you use it differently.
I don’t think there is a best graph, but there can be a best graph for a particular data set and a particular client(s). The way someone makes decisions based on a visual could dictate what works best. That’s my advice: work with your clients.
For good general advice, I think you should lean on https://www.storytellingwithdata.com/, Meagan, and resources such as Edward Tufte, who have spent a lot of time thinking, experimenting, and understanding how to convey information visually. You could learn a lot from them.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I ran across an interesting post from Rita Fainshtein that looked at the different types of graphs for a set of data. I thought that was interesting, so I ran my own experiment. I found for my data, a line graph was better, but let me know what you think.
My data set was simple, a few players across a few events and their number of kills. I coach volleyball and I’m always trying to present stats in a useful way. Here was the small set I picked.
Nothing fancy, but this is similar to what was in the article. From here, I asked EXcel to insert a chart from the Recommended Charts and picked the first one. This is what I saw:
This isn’t bad, though it’s hard to compare across events. I see how A did for all these events, but if I want to see if she was better or worse than K, I’m guessing on some of the lines.
Next, I opened Power BI. I added my data set and then dragged a value to the visualizations. The default chart I got was similar to Excel. Not a great visual. It’s Ok, but suffers from similar problems.
I moved the player and the event from the axis to the legend, because that’s more appropriate for this dataset.
I changed the type to the first stacked chart I saw, and got this, which I think isn’t useful at all. There isn’t a way to measure 100% in this case, so normalizing all data in this way is very unhelpful.
I switched to a normal stacked chart, which is OK, but for Crossroads, is A or K better? Does E outperform A in Aurora? Very hard to tell.
Next I tried a line chart. This I think is the best for my analysis. I can see the events across the bottom and the colors for the players make it easy for me to compare them with each other. I am not color blind, so this works for me.
I can also compare the players individually across events by following the lines. I can see them all trending in similar ways, which is expected. If the team does well, everyone does better. If not, then not. I also see the outlier where E outperformed A.
SummaryThe line graph worked best for me. However, the data I am looking at lends itself to a comparison that looks good here. The stacked bar chart isn’t as clean when drawing conclusions, but that’s because I’m looking at the different segments across items, not necessarily with segments for the same item. Of course, I think that would be hard as well.
Ultimately the data you are analyzing and how the analysis is used will affect what you think is better. Work with the people who use your visual to make a decision and let them help guide you for the best visual.
In this case, A loves seeing she’s ahead almost all the time.
We all experience technical debt when building software, especially software that has been around for years. Most of us have likely encountered code that we are loathe to touch and modify. At least, we don’t touch it twice. We might change things once, but when it doesn’t work as expected, most of us put it back in it’s previous state and move on to something else.
At the same time, it’s hard to define what technical debt is or point it out to managers, who often can’t grasp the concept of why technical debt matters to a development team. They can’t understand the reasons why new code built on older code takes longer and longer to produce. It’s a nebulous concept that can baffle both people new to software development and those with years of experience.
I ran across a few articles that talk about technical debt. This one notes it’s commonly used as a phrase and also, it’s inevitable. There’s a short video from Ward Cunningham, who is said to be the inventor of the term. It’s interesting because the explanation isn’t bad code, it’s more that we don’t always understand the problem when we write code, especially at the beginning. As we have a disagreement between what’s coded and what’s needed, we accumulate debt. Ward talks about refactoring regularly as we gain understanding.
There is another piece that uses the Chernobyl disaster as a comparison with software development. This one looks at cost-cutting in both situations. The emphasis in software development is something Jeff Moden will appreciate: developers not writing robust code. There is also a section on designing to meet only 95% (or less) of perceived use cases, which I think is just reality. We can’t write software, in any practical sense, to cover 100% of use cases.
The final section talks about incomplete testing. While software development has gotten better here, there is still plenty of work to be done, especially with database software. I find far too few database engineers wrap tests around their code and often have problems with new data or when code is refactored. A few more tests added early often prevent issues later, or at least, save time when debugging. This is one place I think AI might really help with labor savings by writing these tests for us.
Technical debt is a reality of life. We have imperfect engineers, we have the pressures to get things done, the increasing complexity of software systems, and perhaps most of all, the pressure to keep moving forward with something new instead of refactoring something old. I don’t have any great solutions, but I also do see software developers working better than they did in the past. That gives me hope for a future where software is embedded in more places and used more every day.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
The code from my talk today at SQL Saturday Jacksonville is available in GitHub in a repo: Zero Downtime
There is a description in the readme, but you can open the DBClient folder and the VS solution for ZeroDowntime in there in VS 2019 and run it.
The SQL Code is numbered in order in the SQL folder.
If you have questions, please feel free to contact me or submit an issue on GitHub. Please feel free to use this presentation at your own employer or usergroup.
I’m heading to SQL Saturday Jacksonville 2023 today, speaking tomorrow. After not making any of their events, I went in 2022 and am back for their 15th event this year. I’m excited to go, though this will be quick trip. Some personal commitments have me going this afternoon and coming back Monday. This was supposed to be a Thur-Tues vacation + work, but it isn’t.
I’m delivering a session on Zero Downtime Deployments, but there’s a bunch of great content scheduled. If you’re nearby, I urge you to register and attend.
Many technical people dream of starting a company and having it grow to an IPO. The history of computing is littered with the successes and failures of companies that attempted this. As a young man in high school, I saw the explosive growth and success of Apple, dreaming of working for them or duplicating their success. In the late 90s and 2000s, I tried a few startups before I founded SQL Server Central with my partners. I had success with that company, though no IPO :(.
I’ve seen many friends start companies, often consulting ones, though a few have built products. Building and running any enterprise is hard, and startups are no exception. There was a post on Hacker News looking for advice and business lessons from others, and I enjoyed reading the responses. I wouldn’t read much deeper than the first level of responses, as many of the subsequent comments are less interesting.
This response was something that caught my eye, especially as someone that’s had to do more than the technical part of running a business. A lot of the challenges I find in startups are business related, especially when technical people are in charge. It’s good for them to understand that core business, especially sales, skills are important. Even more important, home life matters.
It’s easy to assume that your technology matters. Is the bidding technology at eBay better than uBid or eBid or even Craigslist? They are now, but 20 years ago many of these sites were similar, and some had arguably built better systems. Was the tech behind Amazon in 1999 that far ahead of Barnes and Noble or Borders? Certainly, Amazon continued to invest in technology at a scale that exceeded many other companies, but they built their success because of execution in business, not technology.
Starting a company is hard, and it takes a lot of work. Whether you create a product or you are the product. I’ve seen many over the years, worked with some, and often the best products, tech, or people aren’t the ones that succeed. The ones that are best at running a business succeed.
If you want to try and start a software company, go into consulting, or perhaps create another business, I urge you to learn a lot about how to run a business. Basic sales, marketing, and accounting skills will go a long way towards helping your endeavor succeed. For many of you, the tech is the easy part, so don’t spend all your time there.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Chysalism – n. the amniotic tranquility of being indoors during a thunderstorm?
While I love the sun, I did enjoy thunderstorms when I lived in Virginia. Something neat about being inside, cozy, enjoying live while the weather outside has thunder, lightning, and rain creating chaos. It rarely happens in Denver, but I do usually appreciate when it does.
From the Dictionary of Obscure Sorrows
There has been a lot of attention given to ChatGPT and AI over the last month or two. I’ve tried a few things with the public interface at Open.ai. Some worked well, like this one:
Others not so well:
This post looks at a few things I tried with VS Code and GitHub Copilot.
Getting AccessI saw a note in our internal Redgate Slack that all developers were given access to Github Copilot. This is something you can subscribe to for about US$10/month, or if you are a student, you can get it for free. In my case, I filed a ticket:
It took a week or so as someone was out on holiday and this was a low priority ticket. In any case, I got a note from our ticket system, as well as GitHub, that I had access:
So I added the extension to VSCode.
Once installed, I got a note to sign in to GitHub, and when I had completed that, I saw the Copilot icon in the lower right corner of my IDE.
Getting StartedMy first experiment was to open my ZeroDowntime client code and see what happened. This is a VS 2019 project, but I opened it in VSCode, specifically the form1.cs code. I highlighted some code and …
Nothing.
Then I tried something I’d seen. I added a comment above some code. Still nothing, but when I opened the Github Copilot completions panel, I saw this:
Not helpful, nor what I asked for. Both solutions were similar.
Starting from ScratchI decided to then start from scratch. I created a new file, set this to C# and wrote this:
Initially I got nothing, but when I opened the Copilot panel I saw this:
That is more interesting. Clearly poor specifications on my part.
Let’s try something else. I added more detail, and as I did, Copilot even added a few things after to help me specify what I needed.
I finished these lines and hit enter and I got something, though not what I wanted.
Let’s try something else:
That’s better. Not great, but better. This doesn’t quite match what I asked for. I copied this to a template project and then added another comment. I want to check for the existence of the parameter, and I got this code, which is better than something I’d write.
Not quite right, and I have some lines to delete, but this gives me something, and as a middling C# dev, this is helpful. Once cleaned, I compiled and ran the code, and it seemed to work, at least for a very basic console app.
Let’s try SQLI opened a new file, set the type to SQL and wrote a comment, using the appropriate comment style.
I guess that’s OK. Not great, but I didn’t provide much detail. Interesting that it chose to use a created_at column with a timestamp. I never use timestamp, which is deprecated. Not sure how the AI learned about this type, perhaps because of a large corpus of training data using old code with this? Who knows.
Let’s try something else. I’ll ask for a query using a known schema.
A start, but not quite right. I do have to keep hitting enter to get the query written. A couple more Enters and Tabs to accept code get me this:
If I keep going, I get a long query, but it doens’t work. At least not on my version of AdventureWorks.
Not great.
What if I add a comment? I’ll ask for window functions.
I get something else, but again, the final query doesn’t work. This time there are less errors, but the join listed seems to think the Customer table has a first name and last name, which it doesn’t.
Initial ThoughtsI’m not sure how useful this is. I think this is going to be one of those tools that I’ll have to practice with and understand how it works. My basic tests are mostly because I’m not sure what to do with it, or how it can be helpful.
I have lightly seen some demos, but I realize that I need to watch a few more and also experiment with the features. I was hoping it would clean up some of my C# code, which is fairly basic, but it didn’t, at least not with my prompts.
We’ll see how this goes, and I’ll see if I can use it in ADS and if it can actually recognize and use my database schemas to write queries or tests.
It’s my fault.
That’s what I think if there is a security incident with my employer that involves the database. It’s almost my first thought when I hear about issues at other organizations, thinking a technical person is at fault. Since I’ve been a developer and administrator, and I know how complex systems are, I usually stop myself and try to learn more before I assign blame.
The public and your customers also think that it’s just your fault. At least, that’s what I see and hear from friends. Non-technical people are very quick to assign blame and get upset. They can’t understand why some companies get breached and others don’t. To them, it’s because the staff or management are lazy and haven’t done a good job keeping their systems secure.
However, even my technical friends get upset. I’ve had more than a few of them chastise an organization for getting breached when they themselves haven’t always kept up to date on patches. I mean, how many of you are sure every SQL Server you have is at the latest CU level? How quickly do you patch? Are you sure your firewall people haven’t accidentally misconfigured a rule for port 1433?
Anyone can get breached, as noted in this article. However, a good response can set you apart, and I wish that more management and technical people would be prepared now for a data loss incident, a ransomware attack, or really any security issue that might occur in the future.
It’s easy to panic and make rash decisions. The best time to draft your response is now, when you have a clear head and no pressure. Have a few people start to game out how to react, what words and message to send, and who will take responsibility for communicating with customers. It’s worth a little exercise to discuss some possible responses to events and at least have the outline of a plan.
And no matter what, be sure you have a copy of the plan air-gapped from your network. On a few flash drives, saved to a separate OneDrive/Google Drive/Dropbox account, or even printed out. The last think you need is for all of your work to be inaccessible because of something like ransomware encryption.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I play guitar as a hobby. Not great, but I enjoy it and find it relaxing. It’s a good break from my day periodically, killing a few minutes before a meeting. It’s also a nice way to unwind at night. I find it better than playing games or my phone or streaming more Netflix shows. I enjoy those as well, but there is something different about music.
I tend to use acoustic guitars, meaning no electronics. However, I have had electric guitars in the past, and have even used pedals to alter sounds. When I saw this article about design lessons from guitar pedals, I was intrigued. It has 5 lessons from these devices, which are for other digital gear. However, I think they could apply to software as well.
The first one is that these pedals are rugged. While I’m not stomping on pedals with my feed, I do think that we could ensure software is more robust and not susceptible to small mistakes by users, especially in the order of their touches/clicks that might cause problems.
The second is about using more than our hands, which I hope doesn’t apply. You can add voice or gestures but don’t require those. I HATE those features. The fourth is about physical UIs, including physical buttons, which I think is important for cars, but not necessarily for all software. However, if you can give someone a button or knob instead of a touch, it can be helpful.
The third is to have bold, visual cues. I had a designer once say we ought to build more Fisher-Price software, meaning something obvious and usable by a child. I know some of our processes are complex, but we ought to work to keep things as simple as we can. For databases, I think clear, consistent names help here, especially for indexes, FKs, and triggers.
The last one is to make things beautiful. I have to admit I didn’t think about this much before coming to work for Redgate Software. Across the last 15 years, I’ve learned to appreciate the value of design, UX, and the people that make things look good. I can’t do that; I have no skill in this area, but I know that having someone come behind me to do this is worth the effort.
Of course, on top of all this, your software has to work and perform well. If the software doesn’t work correctly or is very slow, none of your clients are happy. Learn to write better code, improve your skills, and listen carefully to those asking for features. If you do that, these design lessons will make sure all your efforts shine through.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I don’t know how many of you use the ScriptDOM. I haven’t really used it, but was very impressed with Mala Mahadevan’s Stairway Series on the topic. I have recommended this to a few customers that were looking for some complex code analysis features, which go beyond what SQL Prompt or SQL Fluff do.
I noticed this week that ScriptDom has been open sourced by Microsoft. The code is available on GitHub, which means you can fork it and change it. Or submit PRs. No idea if Microsoft will take them, but if you write solid, useful code, they might.
I like that more and more Microsoft is open-sourcing and sharing code that they write. Usually, their repos aren’t for software they sell, but maybe they will change that at some point.
There are over 5000 repos in their account right now, including one for VSCode, which I use almost every day. While I don’t plan on contributing or even bug-fixing, I bet some of you might. I might contribute to the docs, which I do regularly for the SQL Server docs. There are a lot of changes here, but there are a few marked way0utwest.
BTW, if you don’t want to do your own PRs, send me a note. I’m happy to edit the docs and submit changes.
I am a fan of open-source projects, because I do think collaboration is useful in many situations. While I don’t expect many people to actually make changes to software, some will. Some, like me, will correct docs, and others will find issues in the code and report them. All of those efforts help us improve software, and I am all for higher quality software.
Now if we could get Microsoft to open-source SSMS, maybe a few of you would find ways to improve that application.
Steve Jones
This is completely off topic and feel free to stop reading if you don’t care about how to tap a bolt, but it was something that I had to a) learn, b) spend time practicing, and c) get done to save money.
tl;dr it’s not hard, but it was interesting.
Bear with me as I’ll explain the situation, what happened, and what/how I learned.
We have this building on the ranch.
That’s not a great show, but it is cool at night. Here’s a better shot.
This is a fabric building, made of a steel frame that’s anchored to the ground and fabric pulled around it. The fabric is actually secured by weaving straps through the frame. Here’s a good shot of how this works, albeit with a broken strap. We actually had to climb up here and tie a new strap to the existing one to secure things. Another skill, but a simple one. Mostly climbing and tying strings.
At the bottom of the the fabric, there is a pocket. It’s actually a loop that runs horizontally along the material. Similar to how a hoodie has a pocket around the hood in which a string is threaded. In this case, there’s a steel pole in there. You can see this below, as the white material at the bottom is doubled over and a different color. There is a small pocket cut out towards the middle left where you can see the steel pole.
This steel pole is secured to the bottom frame with a threaded rod. You can see a couple of these in the image, with a washer and nut on it. These go through the pole and screw into the frame that’s on the ground. The nut is tightened and holds the walls down in the wind. There is some space in some places where the wind and go in and out, which allows pressure to equalize inside and outside the building, and also helps the frame resist strong winds by having it flow through.
I didn’t think much of this when the building was built, but at some point my wife told me a rod had broken and the building was flapping. In this case, part of the wall was moving, which wears the fabric down and can tear it.
Not an easy thing to fix and potentially something that will cost a lot.
RepairsWhen I examined the rod, I saw it had broken off in the base frame. I don’t have a great picture, but this is roughly what I saw, albeit in a round steel frame. This picture shows a bolt head broken, but for me, it was a longer rod.
I did what I often do. I triaged the scope and scale of this problem. I saw articles like this one from Bob Vila. That didn’t look took hard, so I decided to see if I could remove the old bolt with a left handed bit and easily fix things.
I bought a left handed bit and tried and failed. The old bolt is exposed to dirt and the weather and I think it slightly rusted in the nut, or perhaps it was too clogged from dirt, but I couldn’t get it out.
My thought was to call a service person, but I also know this is a small job. Likely someone can do this in minutes, which means it’s not much $$ and hard to get someone to come.
A little more research led me to videos like this one, which shows how to make a mark and drill out the screw. In checking with the costs of the tools, I found some that weren’t expensive, and certainly less costly than getting a handyman out.
So I tried this. I took the broken rod to the store, found it was 1/2”, bought a slightly smaller drill bit (29/64) and a tap/die set.
I got a hammer and a metal drill bit and drilled down into the hole. I used a little gear oil to keep things cool and worked my way down.
Once I had a hole, I read instructions, watched a video, and then tapped new threads into the space. This is really keeping this tool aligned and screwing it down into the hole. It’s hard, and definitely tiring on the hands.
Once I have this through the metal, I usually can easily go up and down a bit, threading it into and out of the hole. This is really only about 1/2-3/4” of metal as the horizontal tube below is hollow.
The last part is threading in a new rod and tightening down the not on the bolt to hold the base.
Reusing New SkillsOnce I did one, I felt confident in doing others. In fact, the first time my wife pointed out the problem, there were 2 to fix. Since then, I find that I need to fix 2-3 every year, in different places. Some from weather, some perhaps from horses or people kicking the base.
I found that after the second time, I didn’t even need to go look up the process, as I internalized what needed to be done.
There are quite a few skills like this I’ve learned in my lifetime, and plenty here at the ranch, that save money, and more importantly, time. They also give me a sense of satisfaction.
On the list this year is to teach my daughter this skill as she’s hoping to come work on the ranch after college and this is something she can do.
The last year has seen a number of large tech companies lay off large numbers of staff. The list for 2023 includes large companies, like Google (12,000), Amazon (9,000 this time), Microsoft (10,000), and Meta (10,000 this time), but also small companies like Zoom (1,300), Rapid (115), and Roku (200). It’s not just tech companies, however, as Disney (7,000), Gap (500), 3M (6,000), and David’s Bridal (9,236) are letting people go. There are plenty of other companies who have let people go, which is interesting to me as the economy has grown in the US, though profits were down. It’s hard to know whether these layoffs are really important for all these companies or whether these layoffs are management’s decision to group their bad news with everyone else’s and take advantage of the opportunity to shrink labor costs.
In any case, layoffs are sad and stressful. Certainly, the people being let go are traumatized and I don’t want to minimize the impact to their lives, but this can be hard for the survivors as well. This isn’t just a Silicon Valley situation, but one that affects many employees all over the world. Whenever there is a large staffing change in an organization, those that remain can be traumatized and unproductive. This is one reason that public companies must notify and disclose layoffs to investors.
This article looks at how some tech company employees react after surviving a layoff, and it reminds me of some of the layoffs I’ve been through. While I haven’t been let go in a layoff, I have had to deal with the aftermath of some friends losing their employment while other friends try to cope. I’ve felt sad, angry, upset, concerned, frustrated, and more. Even as one of the lucky people that kept their jobs, I found myself unable to cope with the changes on the fateful day and for some time after. I struggled to focus during the next few weeks, while also being stressed as I realized the workload grew unexpectedly. There was still lots of work, but less staff to do it.
Anytime you survive a layoff, I think it’s natural to question whether you want to continue working in the same organization. Is business that bad that we need to let people go? Will there be another layoff? Is our leadership actually doing a good job or have they made mistakes by hiring unnecessary people? Am I unnecessary? Are managers appeasing investors who care more about their return or even worried about their own bonuses? All of these thoughts swirl through my head and others’ heads as we move forward. I don’t want you to feel bad here, but to think about your situation as someone that might get laid off or survive one.
Most of us don’t experience layoffs, and if we do, it’s not often that these happen. However, they are always possible, which is why I advocate for all of you to keep learning, regularly grow your skills, keep your resume up to date, and be aware of how your organization is operating. It’s good to work as if you’ll continue in this position (if you enjoy it), but my motto is: hope for the best, plan for the worst.
Of course, if you don’t like your job, you should be working to find another one. The best time to find a new job is while you already have one.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
At Redgate, we’ve spent a lot of time adding PostgreSQL functionality to our DevOps tools. We’ve also hired an advocate, Ryan Booz, to help us spread the word and educate everyone about the platform.
How important is this? It’s hard to know. I know all the major vendors offer flavors of PostgreSQL. Azure has Azure Database for PostgreSQL, AWS has Aurora and RDS, and Google has AlloyDB. I also see plenty of customers using PostgreSQL in some way.
On Redgate.com, there is an article on What is PostgreSQL, and why do businesses need to know more about it? This is a general, high level article that highlights a few things from perspective of business usage. While there aren’t a lot of details, I do find more companies embracing PostgreSQL and other platforms outside of Oracle/SQL Server.
I also noticed a webinar coming tomorrow, PostgreSQL 101: Why PostgreSQL in 2023?, that I am going to try and watch.
I’ve been working with PostgreSQL lightly, and it’s in my Flyway PoC series. I find it both interesting, and in many ways, the same as working with SQL Server. Much of my knowledge transfers, so I’m not worried about learning to use it more in depth if needed.
If you’re worried about your company leaving SQL Server, maybe you want to spend time working with another platform, if for no other reason that you can build some familiarity with tools. However, I don’t know I’d recommend many SQL Server spend time here without a pressing need, and I don’t know that I would advocate for my company to switch. There is a lot to learn, and I think the time spent converting knowledge could outweigh licensing costs.
Who should create documentation for software? In many companies, it’s the developers. In fact, in Redgate, often our developers are tasked with updating articles for products on our documentation site. We do have a streamlined process that has developers can submitting changes in some format (markdown? ) and an automation process that automatically updates the site as part of a release.
However, we also have marketing people who are in charge of external articles and communications. They ensure that our product learning and university content is updated. They work with technical people to produce, edit, and publish content. These could be developers, advocates such as Grant or Ryan, or even partners. Our Friends of Redgate also help with content.
Who should write and update documentation? Developers might best know how software changes, especially for new features. I know that developers also hate this chore, and I think they often don’t have the best perspective because the documentation should explain how I use things, not how I built them or just how I expect them to work. Developers sometimes just want to document how they expect the software to work.
At the same time, marketing or some other area might not be aware of changes, necessitating more handoffs from the development teams. That can be a bottleneck in a DevOps flow. Of course, some of this process could be automated, like sending release notes to marketing. However, there is also the need for someone technical to review changes, which can require more resources. It could also mean marketing people ask developers to review changes, taking up their time anyway. If developers are like me, then they might just want to handle it themselves than communicate changes and still have to review someone else’s writeup.
There probably isn’t a hard and fast rule. Not everyone communicate the same, or even that well. Writing is a skill, and while I might ensure all developers work on this skill, I might also try to lean more on those people that communicate well to ensure docs are updated for our clients.
If you are building internal software, perhaps you forgo documentation entirely. I could see developers having a meeting with either QA staff or perhaps a business person to teach them what is changing. Then this person gets tasked with training others on the software rather than ensuring any formal documentation exists. Tribal knowledge might be the best solution here.
If you’re a developer, would you want to control documentation or avoid it like the plague? Let me know today in the comments.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I had someone ask recently about getting SQL Data Compare results in Excel. It’s easy to do and this post looks at the process.
Exporting a ComparisonI won’t go into the details of making a comparison. I have another post that looks at this with joins, but the tool is fairly intuitive (ingeniously simple) to use.
Once you have a comparison, you should see something like this image. Here I have two tables that are different (I selected all tables). The first, dbo.CountryCodes, has a difference in a row, different values in the name.
The second, dbo.Status, has some rows in the source (left) that are not in the target.
To export these results, I use the Tools menu. There is an option you can see below:
Once I pick this, I get a dialog with options. I can pick certain tables, or all. The defaults are all tables, and only show differences. Note the identical button is not selected.
If I open the folder in the dialog above (after clicking Generate), I see my files. There are separate files for each table and one with a summary.
If I double click the dbo.CountryCodes.csv file, Excel opens, but not the way I like it. I see this:
However, if I File | Open the file, I get the wizard for delimited files.
When I go to the second page and click “comma” as the delimiter, I see a better preview.
I can finish this and I see my data. In this case, the first column lets me know this is changed data that has the same row with the same PK in both databases.
Similarly, I get open the Status table file and see this. Here the first column lets me know this data is only in the first database, the source or left database, that I set in my SQL Data Compare project.
The summary also needs the same open process and this shows me all tables, with lots of zeros. However, for my two tables, you can see there is 1 row noted in the Different column for CountryCodes and 3 rows only in the source (SimpleTalk_1_Dev) database.
I can then save these in Excel format if I like and send them around to colleagues.
SummaryYou’ve seen how you can review SQL Data Compare results in Excel. I don’t know if your Excel will open the CSV with values in separate columns, or if you need to perform a File | Open as I did.
This is useful for sending to a business user that might need to make decisions about what data needs to be synched where. The hardest part here is explaining the _s and _t names for source and target.
SQL Data Compare is very handy for single GB data sets to compare. I wouldn’t recommend this for > 10GB, but under that, with good hardware, you should have success comparing tables or views.
If you’ve never tried it, download an evaluation today.
One of the challenges for both database developers and administrators is doing more, often with less. Many companies continue to grow their database estate, both in width with more platforms, and in depth with more instances of the platforms they have. Some companies will look to shrink their staff, especially when adopting a cloud platform, while others may add more databases, but not increase staffing to match the additional load.
In either case, what many have found over the years is that the cost of labor is high. Both for developers that write code against databases, and administrators that manage those platforms. While licensing can seem to be a large number, compared to the cost of labor, it isn’t usually a significant number.
Often it seems administrators would prefer more of the same database platform. Developers often seem to ask for new types of database platforms, often some type of NoSQL data store. I ran across an article that makes a case for adding in document storage data stores to your environment, instead of just choosing am RDBMS. Labor is one of the big reasons for doing this. The other one is that for a given workload, the hardware cost is lower.
The article opens talking about the object/relational mapping problems. There is some truth to the time and effort to map an object in an application to a table (or set of tables) in an RDBMS. There is some knowledge required to do this, but I also think it’s an important skill for many developers. The same type of object mapping to a serialized JSON document is shown as being easier, and it is.
However, if you add or change your object, the application code to handle the document from the data store gets complex. Over time, you will have lots of “new” fields that don’t exist in older documents. How do you handle those? It’s not hard, but labor is required to write this code. And this code has to be maintained over time.
The other argument is that less hardware is needed, made by noting all the data you may need can be co-located with your object. This is what we would call denormalization in an RDBMS and leads to data duplication? Whether that is a problem or not depends on the amount of duplication. Certainly the structure of an application that often works to send or retrieve singleton rows is easier in a document database.
However, non trivial queries, which the author postulates are hard to write for developers, are likely hard to run for a document database. The load of querying across lots of rows, or updating them, is much higher in a document database. Depending on how often you update data, this can be an issue, and require more hardware.
Which is better? The classic “it depends” applies here. Database modeling is important in both cases. As I’ve worked with people that move to NoSQL databases, I find they struggle to model in that world as much as many of us struggle to model in the RDBMS world. I also find that a NoSQL database often is going to require some sort of data warehouse or other structure that is built for reporting across documents.
I’m not against the various types of NoSQL databases, but I also don’t think they are a panacea of any sort that magically makes building and operating an application easier.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I hosted a webinar a few weeks ago about Artificial Intelligence and how it might affect data professionals. It was an interesting discussion with Kellyn Pot’Vin-Gorman and Brian Randell, with all of us having slightly different perspectives. Overall, we all agree AI is amazing and can be a useful tool for data professionals.
This week I’ve been in Redmond at the MVP Summit, and AI has been a topic among many MVPs. Lots of jokes have been told, no shortage of which dealt with getting rid of staff. Not that these MVPs want to see less staff, but they know that executives and managers might see all the AI hype about how GitHub CoPilot writes code and think they need less developers.
I don’t know if that is really how executives will view the world, especially as most IT departments have more work than resources to complete their list of tasks. I can see AI helping get more done, which might mean less hiring (or slower hiring) in the future.
There was an article this week talking about ways to protect your job in the age of AI hype. It was interesting in that the suggestions all revolve around bringing more value to your job. The suggestions about working in specialized areas, complex areas, being a better employee with documentation, these are all things that I’ve done in my career. It’s what I also see from many MVPs.
Maybe the most interesting item was to think about incorporating AI into your work now. It’s not necessarily an expert on your work, but it is a tool. As the models absorb more information and become better trained, they can be a level to help you get more work done. They can assist you in tedious work, which is something many of us can benefit from.
I am looking forward to getting access to CoPilot and keeping an AI tool up on my desktop, learning how it might help me, and maybe more importantly, where it won’t help me.
I can’t stop this trend, but I can better understand it and learn how it might fit with my daily work. Knowing that helps me better understand how I might have an informed and rational discussion with management about the advantages or disadvantages of this new technology.
Steve Jones
One of the things I haven’t thought about much with my Tesla is range anxiety, even in winter. However, I recently had two almost identical trips with vastly different temperatures. This post looks at what I learned.
This is part of a series that covers my experience with a Tesla Model Y.
The First DriveIn early April I took a day off to go skiing. I used to do this often based on weather, picking sunny and cool days. However, in 2023 I’m more schedule driven. I had to do this particular day because I had the free time.
I woke up and it was about 12F outside. The forecast for Keystone for the day was a low of 1F and a high of 41F. A wild day, so I set out from home. I’d charged the car to 90% the night before, which was a failure on my part. The car had been set to 90, and I had meant to change this to 98, but forgot.
I drove up, stopping a few times for coffee, food, and the restroom. I arrived at the resort parking lot with 35% charge left. It was 7F, and I was a little worried about skiing, but I assumed it would warm up. This also meant I’d used 55% of my charge getting up to the mountain.
It was a nice day, and I enjoyed skiing, leaving around noon. At this point it had warmed up to 34F. I actually showed 36% on the battery driving home. When I mapped going home, the car recommended a charging stop in Idaho Springs for about 5 minutes. Since I planned on lunch from Beau Jos, that was fine.
I parked, walked to the gas station to get a soda and then picked up my pizza. I charged for about 12 minutes, getting 27kW added. More than I needed. I arrived home with 37% left. With all the stops, I ended up going 221.4 miles on 91% charge.
Possibly I could have made it on a single charge.
A Spring Ski DayA few weeks later, I went again. This time it was around 50F when I left home. I arrived at the resort, where it was about 25F, but warming quickly.
This time I’d charged to 96% the night before. I got to the parking lot with 49% left. Around 47% used. When I left for home, same lunch stop but not charging, I felt comfortable. I pulled into the garage with 22% left on an 80F day near my house. This trip was 217.7 miles.
SummaryThere is a huge difference between a 30-40F day skiing and a 10-20F one. The car definitely loses some range in the cold. It’s worth paying a little attention here, but I probably could have avoided Super Charging, or at least done less, if I had charged more at home before the first trip.
In terms of comparison, the drives up were roughly the same, same stops. I used 55% in extreme cold and 47% in cool weather. 8% difference here.
The drives back had a few different stops as I ran different errands, but 36% returning from the mountains in the cold. In warmer temps, it was 38%. One thing to note is that the return trip in the cooler weather was more like the going up in warmer weather. Coming home the second time the AC ran a bit.
I got this car because it had the range that I thought would let me day ski without an inconvenient charging time on the way home. I don’t know other EVs would do that, but with the Tesla Superchargers in Idaho Springs, Silverthorne, and Park Meadows on my journey, I had some leeway to stop quickly. Even in cold weather, and skiing in 15F weather is rare for me, I likely could have made this without stopping.
However, life is life. Sometimes things don’t go your way. The charging stop wasn’t inconvenient at all since I needed a restroom and lunch. I actually got the notification from the car it was charged enough while walking to get my lunch.
EVs are interesting, and they have a different paradigm. I have ceased thinking about range almost all the time, but I do plan differently than I do with ICE cars.
One of the most amazing benefits of working at Redgate Software is the ability to take a sabbatical every five years. One of our staff wrote about this recently, and I found myself reflecting back on mine, as well as thinking forward.
The article notes that many people either learn or travel during theirs. That was somewhat of my experience, where I spent my first one learning skills and volunteering at home. My one-year look back is still interesting to revisit today. Unfortunately, my flagpole base failed in strong winds (sad face) and broke the pole. It’s still on my list to rebuild a new one. I still look back on my volunteer time with fondness and try to get back to Habitat every year.
My second was avoiding travel, since I’d traveled a lot the year before. I ended up with the last sabbatical before the pandemic, coming back to work as our office closed. I stayed home, worked on learning and projects, though I did take a trip to Las Vegas to celebrate my wife’s birthday.
Six weeks away from work seems like a lot. Before Redgate, I’d have thought that this was a huge burden on the employer and fellow employees. However, we’ve had multiple people on sabbatical and we cope. We pick up the slack, and things continue to run. As with maternity (and paternity) leave, it’s not as big a burden as this American used to think.
It is very refreshing, and each time I’ve felt rejuvenated. I’ve been ready to get back to work, talking with Redgate customers and speaking at events. To me, this is a great way to encourage retention among loyal employees, as well as a way that can create more diversity of thought among your employees. Where they travel, the things they learn, even the change of pace in their mind often bring them back to work with new perspectives and ideas.
I just crossed my fifteenth year at Redgate, so I’m due for my third sabbatical. I haven’t thought about it, and I am not likely to take it this year. This does take some planning, both in my personal life and at work, so I have found I usually need 5-6 months to decide on something and get plans in place.
What will I do this time? I’m not sure. What would you suggest? I am thinking to travel this time for part of the trip. My wife and I had an amazing travel time in 2022, and there are so many amazing places in the world that I’d like to visit. I am also tempted to try and fit in some learning as well, perhaps a week spent in some sort of educational endeavor.
No matter what I decide, I am grateful for the opportunity and look forward to another break that helps my work-life balance, balanced.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Data modeling is something that we should all be doing when altering the schema in our databases. I’d like to think that most people spend time here, but I don’t think that’s the case. I think plenty of people think “I need to store a piece of data” and they pick a string or numeric datatype and start stuffing in values. If in doubt, just pick a string. It’s why I think we have lots of dates stored in string columns because that was someone’s first thought.
There was a post recently that talked about storing data in its highest form. It was interesting to me because these are the type of decisions I try to make when designing a table. What is the best form in which to store data? The authors talk about picking not only a type that easily converts, but the fields that make it easiest to work with the data in different ways.
I do think that the aggregations or calculations that we need to perform should influence your data type. If you are measuring something, use a numeric. In fact, in their example of movie times, integer is probably the best type. While many databases and languages have time datatypes, some represent a measure of time (timespan), while others represent a clock (T-SQL time). Either might work for movies, but in aggregations, the T-SQL time will have issues beyond 24 hours. An integer is a better choice, assuming we don’t care about seconds.
The second part of the post looks at multiple values, in this case customer loyalty points earned and redeemed. A simple running sum is what we might store in a database, though the application class might need two fields. Of course, modern software often totals these things for a customer as part of gamification and inducement to engage more, so maybe a data store would also want to store the title earned and redeemed, with a calculation to show the balance.
The one thing that I might add for developers to a post about modeling is the need to consider operations at scale. While using a bit more or less storage often doesn’t matter for any row or any operation on a singleton set of data, when we scale across millions of rows, little things matter. Consider how your data might be aggregated and what happens if you have millions of rows to work on. There a better design decision can out perform a poor one by many orders of magnitude.
That and generate lots of data to test. You ought to know how to quickly mock up a million rows to check your queries. You might have a million rows in production.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I’m off today to Seattle, Redmond actually, and the Microsoft MVP Summit. This is an annual conference Microsoft has run for their MVPs, allowing them to interact and learn from the developers for various products.
I am a Microsoft Data Platform MVP, which means that I’ll mostly be seeing sessions and talking with the developers for SQL Server, Azure SQL, Synapse, etc. All the Data Platform stuff.
The entire event is under NDA, so I can’t discuss anything that happens or what I will learn. At least not this week. A lot of this will come out from under NDA in the next few months, and this event gets me the chance to work with some things before they become public.
It’s also the chance to get to know the developers and product managers better. I’ve become good friends with some over the years, and I look forward to the chance to not only learn from them, but share a meal or drink at some point this week.
A quick trip, out today, back Thursday. Just in time to meet the tractor guy Friday.
Google has launched a version of their AlloyDB that can be installed on-premises. AlloyDB is a PostgreSQL compatible cloud database, a full-managed PaaS service. However, they are giving away a free developer edition and a paid for commercial license that can be installed where a customer wants to run it. The new product is called AlloyDB Omni.
I like Google’s approach here. Obviously Google would prefer lots of their customers would move to the cloud version hosted in GCP (Google Cloud Platform), but they know that’s not realistic. Even for customers that want to move to GCP, the customers might need to keep some workloads on-premises for a period time, so Google is giving them an option to modernize their workload with a new datastore that is PostgreSQL compatible, but running on-premises. Presumably this will be compatible with the cloud version and if customers want to, they can just shift to AlloyDB.
This is similar to what Microsoft is doing as Azure SQL versions, both database and Managed Instance, are mostly compatible with SQL Server on-premises. There are easy migration paths, but since the cloud and local versions aren’t quite the same, it might not be simple to migrate. There are lots of tools to help, but the biggest problem (in my mind), is having a development environment that mimics what I get in the cloud. I need to be able to work not only offline, but with the unlimited CPU I get on my laptop. I don’t want to pay for developers to stand up cloud instances to experiment with code.
There is the SQL development container (running Edge), but that’s a container and I find far too many people don’t like working with the container versions. They struggle with getting their data set in the container, or keeping it up to date. Plus there’s the fact that the local environment seems linked to database projects, which not everyone uses. Especially those of us using Redgate tooling.
I have been surprised in the last five years by how many companies have moved to the cloud. I’m especially surprised how many have performed lift-and-shift migrations to IaaS services after a mandate by management. I’m also not surprised that many customers find they’re spending too much, and that both Azure and AWS realize they need to help customers spend less before they lose them.
The cloud can be a good place for workloads, but you need to plan for it, and often you need to modernize apps, make them less chatty, and write better code against your data services. To do that, I think you need a local dev environment and I like the way that Google is providing that with AlloyDB Omni. I wish we also had a switch to get a local SQL Server dev edition to act like Azure SQL DB.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Well, not one year. More like 18 months, but close enough. I’m writing this at just over 17 months of ownership, from Sept 2021 to Mar 2023.
This is part of a series that covers my experience with a Tesla Model Y.
The HighlightsI never need gas and I have a full tank of fuel every morning. That’s the practical me, the real highlight is that the car is so fast it’s a lot of fun to drive. I stop by gas stations regularly for my diet Coke addition, and the few times I use a gas car, it’s strange to have to pull up to a pump and get fuel. In fact, I sometimes drive next to the little store and then realize I need to move the X5 because it needs gas.
The car is fast. Acceleration is still exciting when I use it. Mostly I drive in chill mode as I don’t need the speed, it’s surprising to passengers, especially the dogs, and I sometimes bounce my own head off the headrest. However, when I need it, it makes merging into traffic easy because I can quickly match speeds with others.
The car giving me access to data and controlling it from remote places is cool. I could start it for my wife when her phone died. I can heat/cool it from the gym or ski slope before I get to it. I have seen the UI change across 18 months to add features, like dog mode, like moving the camera display for turning, music controls, etc. I like this.
Autopilot is great. I use it in certain places, and it keeps the car moving down the road, through bends, letting me watch the road, but not have to focus so much. It’s really nice when I’m tired that it reduces the stress of driving quite a bit, especially over distances.
However, mostly, it’s just transportation that just works. I enjoy driving it, and there are times my wife and I are both wanting to drive it. I miss it when I’m in the X5, or the Suburban, or in a rental. At times, I wish we’d gotten an electric UTV instead of another gas one, but I’m not 100% sure Polaris has figured out the process of building a reliable electric car, both with batteries, motors, and software.
The IssuesAutopilot isn’t perfect. First, it doesn’t change lanes. I could subscribe and get that feature for a price, and perhaps I’ll do that sometimes, but it also has issues. Phantom breaking is much better, but it still flips out in a few places in Colorado. The stretch of C-470 from I-25 in the South going west to Kipling has a few places where the car reads the exit lane speed limit and slows from 70 to 55 quickly.
There also are a few places with sharp curves of cresting a hill where it doesn’t handle things. It also sometimes accelerates and brakes like a new driver, being late to do both. I tend not to use it with my wife because she gets annoyed. As a driver, I can predict these places and I’m ready.
Climate control isn’t perfect. Overall it matches temp well, but the lack of vents in the back, either in the console or side pillars, mean there isn’t great control in the rear. The lack of two fans or vent controls in front also means that sometimes either the driver or passenger has to compromise with more or less fan than they might want.
The front defroster also doesn’t work well in sleet. Most of the time it works OK, but my wife struggled with it in a bad sleet/rain/snow storm. I think this wasn’t engineered as well as much of the car.
No having a rear wiper is annoying. It hasn’t been too much of an issue, but a couple times I wish I had one. It rarely rains in CO, so this might be a bigger issue in other places. The view out the rear is poor as well, which is annoying. With the cameras, I haven’t found this to be dangerous.
In line with that, I wish I had a way to pop out the door handles, like the Model S. In snow, or at night, I can’t always find them easily, especially while carrying stuff. Rather annoying.
It rarely happens, but sometimes the app loses connection to the car and I can’t open the doors. I have to kill and restart the app. Not a big deal, but it does happen. I also don’t always have my keycard with me, so I get slightly worried this might cause an issue at some point.
Lastly, a trim piece in the car broke and was replaced under warranty. It needs to be replaced again
The CostsWhat has this car cost me? Here’s a breakdown:
- Fuel: $919.80 in electricity (ish, my logger was down a bit). This is for 28k miles. My 100mi cost is $3.28. I’ve also spent about $87.44 at Superchargers.
- Windshield fluid – 5 bottles at around US$4
- New Rims – 1058.96 (busted one in a pothole. A single one from Tesla was $400 so I just got a new set aftermarket)
- Winter Tires – $465.18
- Flipping tires – $270 (3 slips. summer->winter->summer->winter)
Of these costs, the tire costs would have been roughly what I’d have done in many other cars. Rims, hard to know. I could have bent one in any case, though the weight of the EV probably made this more likely.
Maintenance has been about $20. Fuel is low. My 100mi cost in the BMW is easily $14-15 or more. That would have been around US$4k. That’s $3k savings.
Registration is adding a new $8 sticker for EVs in my county, which I wouldn’t pay for a petrol car.
SummaryMy wife and I were driving recently after a snowstorm. The car started beeping because it thought I was getting too close to the curb, but I was far away. However, the white snow had piled up and likely reflected enough to worry the car. Annoying. It also slowed the Autopilot during a light snow storm because it couldn’t see. Again, annoying.
However, we both think the car is still the best one we’ve owned. We enjoy it immensely, even with a few quirks. The dog mode alone has been a large value add for us.
This article has a great opening quote. It says: “We are drowning in information but starved for knowledge”. It’s from John Naisbitt, who wrote the book, Megatrends in 1988. I think this quote can be very apropo in organizations as we have data, we have plenty of reports deriving information, but sometimes we don’t have a lot of knowledge, especially when there has been turnover in our staff.
We can train new people on many things, but not everything. The knowledge of the culture, of what others know, of the little strange bugs or behaviors that can’t get fixed, the tribal knowledge accumulated by living in an environment. These are the learnings that can’t easily be replaced, and until they are, often new employees are less productive.
Of course, sometimes new employees will view the world in a different light and find solutions others haven’t considered. That happens, but it’s more likely that they will make mistakes, break something, or just be less effective than their predecessors.
There are no shortage of articles, like the one above or this one, that discuss the concerns IT leadership has about employee turnover. Perhaps the leadership does see that as a problem, but often first level managers don’t. Often leadership doesn’t realize how poorly trained or effective first line managers are in working with their technical staffs. I sometimes think that the world in Dilbert is far too prevalent precisely because of very poor management skills. It certainly seems that efforts made to retain employees are relatively rare in many companies.
In my career, we’ve had some boom and bust times in the market for developers. There have been times where anyone with a certification or a hint of experience could get hired (or get a raise). There were other times when people were careful to hold onto their jobs because finding a new one could be hard.
I think employers should not only invest in their staffs, but work to train and upskill them, demand more from them over time, but treat them fairly with more than just a paycheck. The dividends from a hardware allowance, a training budget, and more can easily pay for themselves with better productivity and the lack of fees to recruiters. You should certainly hold staff accountable and responsible for getting work done, but make sure you also treat them fairly and support them.
That’s if you also spend time managing your managers and ensuring they balance the demands they make of employees with the support needed to ensure their staff performs well. If you ignore the managers, you might as well ignore the staff, set aside more recruiter fees in your budget, and hope for the best.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
In the last post, I created a baseline marker for Flyway in each database. This set the version in the dev and QA databases to v1. However, I also need a baseline script, at least the tool asks for one, so this is the process if you have objects in your production or other downstream databases.
I’ll do this for SQL Server and then PostgreSQL.
Why do this?The main reason to create a baseline script is to note which objects already exist in production. For these objects, I don’t want to track these are changes in their current form.
For example, if I already have a CountryCodes table in production, when I create a project, I want to tell Flyway Desktop that this table exists in production, so if the dev version matches, don’t add this to scripts. If it doesn’t, then I’ve done something in development and need an ALTER script deployed to prod.
What was the Other Baseline?The first baseline in this post, is a version marker. I hate that this is the case, but both Flyway (pre-Redgate) and Flyway Desktop (evolved from SQL Change Automation), had the concept of a baseline, but these were somewhat different things.
Flyway Baseline – The initial version of the database. Don’t deploy any scripts that are <= to this version.
Flyway Desktop Baseline – A script that has the structure and code of all objects that exist in the target database(s).
We can create a baseline script for Flyway, which looks for a B script, but the baseline command expects that you create this script manually. This is used to populate a new database with the baseline migration script prior to running all other scripts.
Setting up the Baseline ScriptFlyway Desktop makes it easy to create a baseline script, and in fact, prompts you to do so.
In my project, if I go to the Schema Model (first) tab, I see there is an object in Development. This was the table I created when I set up the database. The goal is to get this table to other environments.
This table doesn’t exist in QA. I do have the flyway_schema_history table, which was the result of the baseline command.
If I go to the Generate Migrations (second) tab, I see this. The first thing that the tool wants is a Shadow database.
The shadow is essentially a development V-1 (v minus one) version. This is where I test all migrations, compare the state with development, and then determine what’s changed. This is just a regular database, but I create this outside of Flyway Desktop. For me, I created a database (FWPoc_1_Dev_Shadow) and then clicked Set up shadow database to get this dialog. You can name this anything.
I enter details, and test the connection before saving this. In general, this ought to be saved to my user settings as I’ll have my own shadow different from other developers. I DO NEED to click the “ok to erase data” box.
Once this is done, I now see another prompt on the Generate Migrations tab. Now I need a baseline script. I don’t have anything, but I will click the button.
This gives me a dialog to pick a target database. This target is used to get the initial set of objects to populate in the baseline script. You can use production or a copy (recommended) as the target database.
My QA is the same as prod, so I add that with the proper connection string and then I see the target here for the Baseline. I am ignoring static (or lookup/reference data for now). I’ll click the Baseline button.
This runs and … nothing.
Which makes sense, as there is nothing in my target database. I actually get an error after this, which tells me that it doesn’t make sense to baseline an empty database.
I wish that were surfaced earlier. In any case, if I close this, I get the same image above, saying I don’t have a baseline. For now, I’ll ignore that.
SummaryNot much happened in this post. I added a shadow database, which I’ll use to generate scripts. I tried to baseline, but that errored, as it should. I really don’t need a baseline, so I’ll come back to this later in another format.
For now, I’ve advanced the SQL Server project. I’ll actually repeat these steps for the PostgreSQL one, but it’s really creating a new database for the shadow and setting a connections string. Everything else looks the same.
The next post will actually generate a script and deploy this to QA.
Companies often want more data to help them make decisions on how they run their business. There has been this quest to gather and analyze as much data as possible to increase the efficiency of their operations to help reduce costs or increase profits. This has led to the importance of data as an asset, and the need for more data professionals in many organizations.
That’s good for many of us that work with data.
However, using data to try and improve your efficiency has a downside. It can lead you to a very narrow focus in your approach. That can be good in narrow, well-defined areas, such as minimizing the distance driven or packing containers. For less focused tasks, such as telling a story or writing code, this can mean you get stuck in a rut and limit your opportunities to improve.
There’s an interesting article about big data and Hollywood, specifically looking at the types of products produced. Big data analysis leads companies to aim for the most effective types of movies that make money. Good for a company, not so good for society. Arguably, not even good for a company over time as people will tire of the same story, or type of story over time. Eventually, making simple decisions based on past data will start to fail.
I can see the same thing in other industries as well. Using Big Data to drive decisions can help, but many of the areas where we use these techniques will evolve and change over time. The way we solve problems with code change over time as we develop new tools, techniques, platforms, languages, etc. There isn’t a perfect way to design a database or write a CRUD app precisely because new capabilities or new possibilities emerge. You could say the same things about marketing, manufacturing, medicine, and many other endeavors.
This isn’t to imply big data and complex analysis isn’t helpful or useful. It’s just not everything. We need to balance human input, with some creativity, some instinct, some diverse thought, and some guessing. Most importantly, we ought to experiment and learn, not only from what machines might extrapolate, but from how humans change their thinking over time.
Find a balance, accepting some imperfection in your process and in the world at large. Hopefully that will lead you to some success.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I delivered a session on graph databases, and in it I used RedisGraph to show how you can work with graph data on that platform. This blog shows a basic creation of a graph in RedisGraph. This isn’t intended to be a comprehensive post, but more a basic look at the code to work with graphs.
Adding DataI connect to Redis with the command line. If you don’t know how to do that, I covered this in another post. Once I’m connected, I can run this code to create a graph and a node:
GRAPH.QUERY Northwind "CREATE (:employee {employeeID: 10, firstName:'Steve', title:'President'})-[:REPORTS\_TO]->(:employee {employeeID: 11, firstName:'Tia', title:'CEO'})"
This will create a graph, two nodes, and one edge between them. This starts with GRAPH.QUERY, which let’s Redis know this is a graph query. I then give the name of the graph, which is Northwind since I based this on the Northwind dataset.
Next I use a Cyper language CREATE and enter the data in what is very similar to JSON. I label a node (employee), add the properties and values (employeeID and firstName with 10 and Steve as values).
I use the square brackets with the edge name in between this and another node. This will get me the results shown below. You can see that one label, two nodes, six properties, and 1 relationship created.
I can get the employee nodes with this code:
GRAPH.QUERY Northwind "MATCH (e:employee ) return e"
This returns the two nodes.
If I want to see the relationship, I need to ask for that with this code:
GRAPH.QUERY Northwind "MATCH (e:employee)<-[:REPORTS\_TO]-(sub) RETURN sub.firstName as employee, e.firstName as manager"
This returns my graph as a series of data elements.
Tada!
There is more you can do and certainly you can create more complex (or larger) graphs in Redis. The drivers from various languages should return JSON to you that you can deserialize and visualize or otherwise process results.
A lot of this is basic, but if you want to experiment with Redisgraph, a container and the CLI is a good way to get started.
No, Redgate Software isn’t hiring for SQL Prompt. I’m sure quite a few of you depend on SQL Prompt and would like more engineers working on it. Maybe a few of you would find that an interesting piece of software to work on.
Rather, a Prompt engineer is someone that works with AI, trying to get a system to produce better results. I can’t decide if this sounds like an interesting job that stretches your brain or the equivalent of a mediocre developer that just keeps copying something from Stack Overflow, hitting compile, getting an error, and repeating that cycle.
I can picture a scientist using their voice to continually correct some AI system by saying “no, that’s not quite right. I want to see more blue” or some other type of guidance. Is this job the equivalent of raising a digital toddler?
I actually found one of these jobs on Indeed.com. For a company in San Francisco, a hybrid position paying between US$175k-335k to help create steerable AI systems. I’m not entirely sure what this is, but at that pay rate, it is tempting to learn.
I think AI systems are fascinating, especially for someone that has done some development and can see how difficult it would be to specify an algorithm that can handle some tasks. Like composing some prose result. Or suggesting code based on what you’ve typed. These are some handy features that have been added to some IDEs and services.
GitHub CoPilot is interesting, and if you haven’t tried it, I think it helps Java/C#/app programmers. Go watch some videos or sign up for a trial. I am not sure about these helping us in database work. There is probably a little more work needed before an AI delivers help for your particular database issue. However, I’ll be testing and working with some tools to see what I think.
If you are interested in AI and have a lot of patience to train models, and a lot of patience to work on data cleansing and loading, then maybe you can find a new career in AI. It might be a fad, but I don’t think it will ever go away, and I do think there will be a lot of opportunities in this space in the next decade.
Note: If you find this interesting, I had a discussion recently with a few others in a webinar.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I set goals at the beginning of 2023 for Q1. I didn’t do well, as my evaluation earlier this week has a D for my efforts.
I’m re-evaluating things, and for now, I’ve decided not to set more goals for the time being.
Why?One of the things I end my career presentations with is this: we work to live, not live to work. Find balance in your life, between your career and life.
I truly believe that, and as I re-evaluated myself last week, I realized that I am out of balance. There have been some stressful things in life that have been hard on me, but more than the actual thing, it’s had an impact on my mental health. I’ve been a bit drained, so I haven’t wanted to take time to focus, preferring to relax a bit more with my wife, guitar, or a good book.
I started to recognize this when I did my Feb update, but thought I could push through. I didn’t, or couldn’t, and while I appreciate the optimism, I realize that I don’t want to ignore the additional stress in my life.
A Deliberate PauseI don’t want to let goals go, but I am going to ignore them for Q2 and focus on being kind to myself and recognizing I need balance here. I’ll revisit this in late June and see how I am doing then.
There is a survey from the WIT group for female speakers. If you are a woman and speak in front of groups, or used to, please fill it out.
This is part of their efforts to help encourage more females to speak and support their efforts. I like the fact that they are reaching out to others to learn how women feel about speaking.
If you know any women that speak, whether in the data community or elsewhere, please pass along the link.
There was a Slack thread at Redgate recently where a developer was showing some code where they decided to use the “extra” column from the information_schema.columns view. They were making decisions on how to detect certain metadata about a column based on the data in this column. Apparently, the data in here is overloaded for different options that might be set on a table.
This caught my eye because I had no idea there was a column named “extra” in this view. I flipped over to SSMS and decided to check what was being stored in here. To my surprise, there was no “extra” column. As I dug in a little deeper in the thread, I realized the developer was talking about Information_schema.columns in a MySQL database.
That was a surprise to me. While I know different platforms will add features and functionality to their databases, I thought the information_schema views were consistent across platforms. They should give you a set of information you can count on. Apparently, that’s not true. You can count on some things, but not all, which means that these aren’t consistent structures.
Perhaps it doesn’t matter. It seems every product out there will extend the SQL “standard” where they see fit, adding features or functions that suit particular use cases. Commercial vendors do this for profit, and OSS projects likely do this because an individual wants a change. That has resulted in a wide variety of database platforms that meet different needs and solve different problems.
It would be nice if we could write SQL code and be sure it would run on SQL Server, Oracle, Snowflake, PostgreSQL, or any platform. And in many cases, we can. Lots of basic queries are the same. However, what would be the point? I certainly don’t want more people in management wanting to switch from one platform to the other, just because they feel like it. I’d imagine that we’d thrash between platforms every time a senior developer or VP decided a system should run on their favorite platform.
A base standard is good, like a base class in programming. However, they aren’t always as useful as they seem, and extending them to meet needs is better for us all. I don’t need a standard implementation of the SQL language or the information_schema views, it was just a surprise to realize that this actually how the platforms are coded.
Note: If you find this interesting, I had a discussion recently with a few others in a webinar.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
You can register today for the 2023 PASS Data Community Summit. This year the event is in Seattle, Nov 14-17, and in-person only. The event was a lot of fun last year, with many great sessions and lots of networking with fellow data professionals.
I’m looking forward to this year’s event. I have submitted a session, though I am hoping for less of a load this year. Last year I had a session, hosted a panel, MC’d two lightning talk hours, was a part of two keynotes, and I’m sure I’m forgetting something. This year I hope to have a few less commitments and see more sessions and friends.
There is a lot to learn, and this is a great chance to talk with many of the SQL Server developers from Microsoft. With the event in Seattle, I expect a lot of them to attend. It’s also the chance to talk to many of the experts in the community. People like Erik Darling, Allan Hirt, Glenn Berry, and more.
This is the cheapest rate you can get now. They’ll be a few price bumps, and there might be some discount codes, but you can register for a three day conference for less now. Talk to your boss, get some funding, make a case for the ROI as well as suggestion that sending you helps retention.
I hope to see you in Seattle this November.
The grade for March is also D. Details below, but just not making a lot of progress in these areas. In fact, I find myself not motivated to work on goals very much. That’s a bit sad, but it’s also where I am in life.
So far I have for 2023:
- Jan – D
- Feb – D
- Mar – D
I set goals at the beginning of the year, and I’m tracking my progress in these updates during 2022.
Reviewing GoalsThe last month has been hectic. Both life outside of work and work have had me tackling lots of little tasks, which is distracting and reduces my energy for focused work. As a result, I haven’t been doing much on these goals.
I think I’m slightly overloaded, and worried about burnout, so I’m going to ignore goals for Q2.
ReadingTwo books on the list: marketing and coaching. I haven’t picked a marketing book, but I picked Wolfpack for coaching. With one started and one not, this is an F.
Progress:
- Wolfpack – 47%
CareerMy goals:
- Set up a tracking system based on the book for my efforts with customers Track and calculate my scores to help me better approach how I work with customers. Review this with my boss and with someone in sales
Rating: B
I have been tracking my interactions with customers here. There haven’t been a lot, but it is interesting to stop and think about how this is going. I have chatted with my boss and gotten a little feedback.
CommunityHere are the goals I set.
- 2 speaking engagements for community (RMOUG Training Days complete. SQL Bits complete) Reach out to 10 SQL Sat groups that did not run an event in 2022 and motivate them for 2023. This should be at least 2 emails to each person/group. Reach out to all 2022 organizers and ask about plans for 2023 Start coding a tool for converting schedules to HTML. Hold office hours once in Q1* Send 3 monthly SQL Saturday updates to the community
Rating: D
I did complete another community (unpaid) speaking engagement. I’ve reached out to a few SQL Saturday groups and we have a few more events on the schedule, but I haven’t had a lot of energy to do this and the limited responses are discouraging.
Yes, I get discouraged.
Haven’t started a tool, didn’t hold office hours, two updates sent.
PersonalMy goals:
- Use my Power BI report for stats Update the Power BI report based on feedback from athletes Build 6 wooden coasters – I haven’t made enough time for my hobby here, so I’m going to start small. We need more coasters (as noticed over the holidays), so I want to build 6. Same style, different.
Rating: D
I haven’t updated the Power BI report. In fact, I found a couple errors in calculations since DAX is a weakness of mine. As a result, I pulled down some pages. I also think my data model is slightly broken and needs work. No time spent on here.
It’s been cold, and when it isn’t, I’ve had other things to tackle, so no time on wood projects for me. I feel bad about this, but I also know I have too many hobbies. With my Duolingo language stuff, guitar, and then coaching, not a lot of time to spend here.
I travel quite a bit every year. Over 20 trips in 2022 and five trips in the first quarter of 2023. To make life easier, I have a few routines that I use to ensure that travel goes smoothly and I don’t forget things. One of those routines is using a parking service near the airport.
This company used to have a fairly manual process, though it improved over the years. The pandemic forced them to move to more contactless service, which I appreciated. I could make a reservation online, get a QR code, and use that to both enter and exit the facility without interacting with anyone or handling money. A bit safer, but the big win for me was that this process was quicker for me move into and out of the lot.
This service worked great in 2021, but sometime in the spring of 2022, I was making a reservation on the mobile app on the way to the airport. After completing the form, I clicked submit and got an error. I wasn’t sure what to do, so I double-checked everything I’d typed and resubmitted.
Again, an error.
I tried a third time, feeling a bit frustrated. I’d stopped for coffee and needed to start moving to the airport. For some reason, I decided to check my email. To my surprise, I found a confirmation of the reservation. Actually, I found three, which necessitated me asking for refunds for the two I didn’t need, while then trying to ensure I actually used the correct QR code to get in and out.
Since then, I’ve used this mobile app multiple times to make reservations, and it always errors but sends a confirmation. I’ve sent a note to the company, but nothing has changed. The web app doesn’t seem to want to work correctly either but has different problems. I’ve tried a couple of other services, but I like this one. I just need to remember to make one reservation, ignore the error, and check my email.
The tech is broken somewhere. Yet it works. It’s mildly annoying for me, perhaps much more annoying to others. This might dissuade new customers from using the service, though the parking lot seems fairly full most of the time. It’s the kind of thing that I, as a software developer, would want to fix.
It’s also the kind of thing I could see management not caring about, and instead asking me to focus on new features or other bugs that are preventing customers from using the service.
There is often more work queued up for software than there are time or resources to tackle them. When anyone is building software, they are constantly making choices about priorities and focus. What do I work on? What should be done first? What bugs need fixing and what bugs can we live with? Working for a software company has helped me keep perspective on the larger picture for a business.
At the same time, I feel the frustration of a customer when things don’t work as I’d want them to work. Especially when an error is involved. This seems like it should be an easy fix, either catch the error and do something, or at least swallow it from the customer perspective. However, I have no idea how widespread this error is, or if I’m the only one for whom it doesn’t work. A good DevOps process would have instrumentation and monitoring to learn the scope, scale, and criticality of this, and other, bugs.
Either way, it’s been alternately annoying and humorous to me. It works, and I live with it, sometimes amused that it’s still occurring. Perhaps I’ll even miss seeing the message when or if it gets fixed.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I saw a very interesting blog post this week from Brent Ozar that asked if SQL Server 2019 More CPU-Intensive Than SQL Server 2016? You can read the post, but a client was having CPU issues and thought everything between their SQL Server 2016 and 2019 environments was the same.
Brent decided to test this and found that on identical systems, he has queries taking more CPU on SQL 2019 than 2016. A few commenters tested as well, with similar results. Not everyone had similar results, but most did. You can get the scripts from the post and try it yourself if you have SQL 2016 and 2019 instances.
One would expect that more recent versions would run faster, or use fewer resources, for equivalent data sets and hardware. I know that’s not always the case, but it ought to be the case for lots of workloads. If not, then arguably the newer version isn’t better. It likely isn’t worth more money, and definitely needs more development work. This is my view of Windows 11, which seems to have returned to the habit of earlier Windows version of requiring and consuming more resources than its predecessor.
I don’t often benchmark or evaluate SQL Server version. I don’t have to make those decisions, but I have seen SQL Server continue to improve on the TPC-E benchmark. However, this isn’t necessarily the same hardware. In fact, across versions, it likely isn’t. There could be more CPU consumed by the same queries, masked by hardware advances (and falling hardware prices).
Is SQL Server using more CPU in newer versions? I’ll let you see if that’s the case on your systems. Even if it isn’t, you might document some queries (in addition to Brent’s) and record the results. That might help you decide when you upgrade.
Steve Jones
There’s a video of Bill Gates taking a drive in an autonomous car around London. I’ve been to London dozens of times, ridden and cabs and Ubers, and even driven a few times. It’s a crazy environment, and I struggle to process everything as a human. This is certainly a challenge for a car.
It’s not a Tesla. In fact, I don’t see many Tesla FSD (full self driving) videos in the UK. I know they don’t have all the updates the US gets, but I struggled to find any actually in London. A few that looked like they were close to London aborted in city driving.
In any case, this is a neat video, and if you look at all the things happening, as a driver, you can see the complexity of navigating around London. Imagine programming this.
If the video doesn’t tender, the URL is here: https://www.youtube.com/watch?v=ruKJCiAOmfg&w=560&h=315
This is a car from Wayve, and it is impressive. Not the car, but the driving and control are something. It’s something I’d like to experience, since I could imagine this being a very neat thing in witness. Certainly would be better than the Uber I took in Las Vegas a few years ago.
They’re hiring. Not for me, but maybe for you.
I give talks on branding and managing your career at various events. Often I am helping people to better market and sell themselves if find new opportunities. One area that I talk about for data professionals is the online profile. I think having one is important, especially as many of our contacts with potential employers are made digitally.
I ran across an article that focuses on LinkedIn, specifically the profile you create there. It has some thoughts on recommendations, accomplishments, certifications, and more. I like the thoughts, and I do think they help showcase who you are as a professional working in some field. It is worth reading through the article and adjusting your profile to include these items.
I’d guess that most people don’t have these things to add. They might have friends who will write a recommendation, but what about classes, certifications, projects, etc? Getting those is real work, and it’s an investment of time and effort to grow your abilities.
Having a full (or fuller) profile helps you stand out. It helps give hiring managers confidence that you are a person who can do the job they need. Getting this full profile, however, is something you need to do over time. Invest in your skills and showcase this in your profile.
One easy way to do this is to take the work you do and document it. Write blogs, share posts, create an ongoing commentary of what you do at work. When you feel proud of something you’ve done, find a way to add it to your profile. This will create the impression that you are getting things done and solving problems at work.
I’m sure all of you do this. Take a little time and ensure the next person that might consider hiring you, or the person you want to hire you, knows this as well. It can be easy to talk about these things in an interview, but you need to get the interview first. Your resume/CV and profile are how you get the interview, so be sure you are taking care of them across time.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
For much of my career, I’ve run SQL Server Central. A large part of the popularity of the site is from the forums, where people can pose questions about their struggles with SQL Server and get answers from the community. There are also some off-topic forums, where people discuss various things outside of databases. In here, we have discussions a about life, sports, and more. While we do expect people to maintain an air of professionalism and respect others, we don’t try to moderate content.
That’s how much of the Internet has worked, with various sites allowing users to post content, but not having any responsibility for what has been posted. The liability for that lies with the person doing the posting, which creates a thorny issue when users post anonymously. Setting that aside, I’ve been a proponent of this, not believing that Facebook or LinkedIn, or SQL Server Central ought to be liable for what users write and post. I do thinks users bear that responsibility.
However, in the US, there is a Supreme Court case that may change our view, and that of many others. This case deals not with the data itself, but rather the algorithms that might display or recommend some of that data to others. That’s an interesting approach to the case law that has shielded many tech companies from their users’ poor behavior. Essentially the plaintiffs argue that Google and Twitter bear responsibility for their algorithms, which in this case aided terrorist recruitment. Meaning that the code they wrote to analyze data, essentially the queries that promoted content to users, were harmful.
There are four possibilities listed in the article for what could happen, and I find them fascinating from a data analysis standpoint. Essentially a ruling against tech companies could shape how many of these companies process data in the future. While we might like to ensure these companies do not promote harmful content, think about this from the data analysis view? Do you want these companies to moderating how they provide results? Would this mean that we need to more carefully craft our search terms? In the context of tremendous floods of information, we often depend on Google, Bing, or some search algorithm to distinguish among the various meanings of words to bring back results relevant to us. At the same time, we might wish that everyone got the same results from the same search terms.
Separate from the results, what about related results, or suggested items that might be related. I find the quality of these can vary for me, but often there is something “sponsored” or “I might like” that is helpful to me. Or just interesting. The infinite scrolling that many people live, getting similar recommendations is a double edged sword. It can increase learning, pleasure, etc. It can also send someone down a rabbit hole of anger and reinforcement of negative emotions. I think this also is one way that the content of the Internet creates division and disagreement among many.
While I think users are responsible for their words, I also think that the way that these companies recommend and showcase content likely bears some responsibility. At the same time, I can’t imagine how you regulate this, and I do not want to see a constant battle of lawsuits over how we interpret rules. The sex, drugs, and rock and roll issues of the past, where we tried to legislate morality, didn’t work well. I don’t want to see that again.
There isn’t a good answer here for me, and of the four possibilities, I fall somewhere between two and three. Some changes to section 230 (the legal writing) but not heavy changes or an abandonment of the way this has been interpreted. What do you think? Should we start to hold companies responsible for how they present content? I don’t know I worry for SQL Server Central, but it might change other sites. For us, we just show things from the last 24 hours. It’s not much of an algorithm, but it is one that likely isn’t going to get us sued.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
While working with a customer recently, I saw some code from them that used the OUTPUT and INTO clauses of an UPDATE statement to capture the changes made into another table. In this case, as users updated code strings in a table used in dynamic SQL, the developers wanted to capture the history of those changes in another table, giving them a rollback strategy if there were problems. This is an interesting, and focused way of auditing data changes, albeit with the work to write this code and control updates through stored procedures.
This is one way of tracking the history of data values in many database platforms. Temporal tables are another, and we have CDC/Change Capture available in SQL Server. In the past, many people have used triggers. All of these work, with various pros and cons. In many cases, I have seen triggers used, often because many developers know how to write them and they are easy to create. Easy to get wrong as well.
However, triggers take some work, while platforms have often built capabilities that make it easy to capture data changes and track them. Many developers often aren’t aware of these features, or haven’t spent sufficient time with them to know how to work with them or if these features even work well. This is one reason many of us write about new features, to learn about them, experiment, and help others to understand how to use them. Of course, not all features turn out to be as great as marketed.
Today I wonder how you capture changes and audit them. Do you use triggers? Something built in? For limited auditing, and with control of the code, would you use the OUTPUT clause with your insert/update/delete code? Actually, I wonder how many of you would even consider this for limited auditing, especially with the large number of tools and frameworks that might generate their own UPDATE statement rather than call a stored procedure where you control the code. Or do you think this isn’t a good way to capture this data.
Steve Jones
As I work with more and more customers at Redgate, I see some interesting trends. During the pandemic (and prior), we got a lot of questions on zero downtime and how to achieve database DevOps without causing problems. Those are always interesting discussions, and I find many people want magical solutions without having to change the way they work.
The last year, however, has had more people looking to implement database DevOps and speed up their development, but not a lot of questions or demands for zero downtime during these deployments. I find that interesting as the world depends more and more on computer systems, and the customer base for many organizations may demand access to the systems at any hour of the day or night.
However, it doesn’t seem that as many people are concerned about small moments of downtime. Does this mean that more organizations aren’t measuring uptime anymore? Perhaps the interruptions caused by software deployments aren’t being counted? Or maybe the application software has gotten better at hiding blips in database access. Perhaps feature flags are catching on as a standard practice, so database deployments are less troublesome.
I’m not sure what has changed, but it has been noticeable by me that the importance of making changes without downtime has not been a requirement from many customers. Is that the case for many of you reading this? Are you less concerned about downtime? I think one nice thing about the move to the cloud is it’s a little less stable, and perhaps that has lowered some of the expectations of our management. Since it’s out of our control, maybe we shouldn’t be too concerned about the need for retries, either automatic or a customer pressing a button again.
Let us know today if you feel pressure to get closer to zero downtime, either in your everyday management of databases or during deployments. Or maybe tell us if you’ve gotten so good at your job that no one every notices when you do make changes.
Steve Jones
Today’s coping tip is to discover the joy in the simple things in life.
This is the last coping tip for now. It’s been 3 years, and I hope you’ve enjoyed them.
I have really tried to enjoy simple, little things in my life and travels. A text from one of my kids. A short chat with an athlete I coach. A conversation at a SQL Saturday or other event. The chance to share a picture, or see one, of a friend.
Simple, little things make life interesting and wonderful.
When I finish a book I enjoy. A small success in a game. Learning a new song or lick on guitar, or even having a single song play well. A tasty meal. A laugh from my wife.
Looking at the world and enjoying little things helps me cope with the hard things, or the upsetting things. Remembering little joys, or noticing them, can reset my attitude or dampen other negative emotions.
Try appreciating and enjoying something small. I bet you feel happier in the rest of your life.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I have been experimenting a bit with graph databases, trying to learn more about them. One of the platforms I wanted to experiment with was RedisGraph. This post looks at getting this set up and running on a Windows machine.
Things You NeedA list of things:
- Windows (I used Windows 10 and 11)
- Docker Desktop
- Redis
These are what I used.
Getting SetupI won’t go into getting Docker running, but you should ensure you have Docker running and set to Linux containers on Windows. Once you’ve done that, then you can start up a container.
You also need to run the Redis install to get the CLI to connect. I installed this and then set Redis to start manually and not start. You don’t need this running, nor do you want lots of extra services going. Plus, you’ll get a port conflict with instructions below.
To start the container, run this code from a command prompt:
docker run -p 6379:6379 -it --rm redislabs/redisgraph
This does run the container with output to the command line, and you should see this when you run this:
To stop it, CTRL+C will stop the container.
ConnectingTo connect, we will run the Redis-cli.exe program. We can connect from programs, but for testing, let’s work at the command prompt. In a second command window, you will run this. My exe was in c:\program files\Redis, as you see below. Yours might be different, but once you find it, change to that directory.
If you run the redis-cli, you see this, a connection to the local host, with the port:
In another post, I’ll create a graph, but to test this, if you type “ping”
Thanks to everyone who attended my talk at VS Live Las Vegas 2023 on graph databases. I hope you enjoyed it, and if you have questions, please feel free to reach out.
ResourcesHere are the resources:
- Slides : https://voiceofthedba.files.wordpress.com/2023/03/2023_vslivelasvegas_th15_graphstructures.zip
- GitHub: https://github.com/way0utwest/AddingGraphStructures
- Neo4J Desktop: https://neo4j.com/download-neo4j-now/
- RedisGraph: https://redis.io/docs/stack/graph/
I delivered my talk on Architecting Zero Downtime Deployments yesterday at VS Live Las Vegas 2023. It went fairly well, even though I ran some incorrect code somewhere. Apologies for that, but glad I could fix things.
The code for the database and the C# app is in this Github repo: https://github.com/way0utwest/ZeroDowntime
The PPT is also in the repo, updated today.
If you find issues, or an improve my C#, please feel free to open an issue or submit a PR.
Today’s coping tip is to focus your attention on the good things you take for granted.
My life is great. I used to say perfect, but a few struggles lately have me appreciating it more and understanding that there are things I wish were better.
I have taken for granted my ability to walk, to use my hands, and to use my mind. Lately I’ve been appreciating those things more as I’ve run into a few friends or friends of friends that aren’t as lucky.
I had ankle surgery last year, and my wife this year. We struggled to get back to normal, and I’m still not, but I am better than I was. I appreciate my ability to walk on my own and hope to get in a lot of walking and visiting new places while I can.
My hands work, but a few years ago Rush retired from performing. Alex Lifeson’s hands were one reason. I think about that often as I try to learn new songs on guitar.
I also have run into a few people who have had health issues affecting their brains. My father was starting to suffer from dementia before he passed. I appreciate my cognition for now.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I am not a great software developer. I’m OK, and I do know how to use Google and Stack Overflow well. Maybe my best skill is wording searches well? In any case, I’ve had to write a bit of C# lately to build an app for my Zero Downtime talk.
In no way am I an expert on this stuff, but I learned a few things while working on the client. One of these was how to get to/from various datatypes. Not a complex skill, but since I do it rarely, I decided to do a quick blog.
Creating StringsI actually remembered how to take a number and convert it to a string. The ToString() method works on many items. In my case, I used some variables as numerics, often a zero or one, but I wanted to display these in a textbox. To do that, I needed something like this:
txtRandom.Text = num.ToString();
Easy enough to write, but other items were more complex.
Strings to IntI only had to go the reverse way a few times, but found there was no ToInt32() or similar. I was hoping for one, but a little google query helped me realize this is what I needed?
cmd.Parameters.Add("@year", SqlDbType.Int).Value = Convert.ToInt32(txtYear.Text);
In this case, I was adding an integer parameter from a value in a textbox, which was a strong. The Convert class has a ToInt32() method I used.
String to DateIn one demo I move to a date as a parameter, again getting a value from a string textbox. This was yet another method. In this case, there is a DateTime class with a Parse() method. I used it as such:
cmd.Parameters.Add("@start", SqlDbType.DateTime).Value = DateTime.Parse(txtStart.Text);
This worked great.
NULLsOne other item I needed was passing a NULL value to a proc. Part of zero downtime deployments involve staging your changes, and that sometimes means altering which parameters you use and sending in other values. In this case, I found a DBNull class with a value property. I passed null parameter values like this:
cmd.Parameters.Add("@year", SqlDbType.Int).Value = DBNull.Value;
MoreThere are obviously other conversions, but I’ll learn those as I need them. I know how to phrase a quick question and read through StackOverflow to find code I need and modify it. I’m good at asking questions and listening.
If you have advice, please leave me a comment. I had some fun doing this, and I’m glad I didn’t need to bother too many people to get this working.
Today’s coping tip is to appreciate your hands and all the things they do for you.
I make a living with my hands. I type constantly, which is a big part of the work I do at Redgate. When I have a cut on a finger, as I did recently, it’s hard to type. That makes me much less productive, and lengthens my workday or workweek.
I also use my hands in job #2. I coach kids in volleyball and I’m regularly hitting balls in the air, which becomes painful or less controlled if my hands have issues. Or my shoulder, but that’s another post. I also take stats on paper and type totals into a spreadsheet, so yeah, hands are important.
Around the house, hands help me fix things on the ranch or play guitar.
I really appreciate my health and ability to use my hands to make life better.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to listen to a piece of music without doing anything else.
This is surprisingly hard. I turn something on, but then I look around, think of something else, walk, exercise, drive, etc.
Travel is a good way to do this, and I’ve learned sometimes that I can take a few minutes to unwind by staring at the world and listening to something in headphones. Hard for a multitasker like me, but it can be refreshing. If I’m not too busy.
For a trip last week, I got to the airport a bit early. I knew I’d eat on the plane, so I went to the lounge, got a glass of wine, and sat facing a window. I put on a little John Mayer and then sat there just listening.
Lately a couple songs have been running through my head, so I ran Hey Marie and Slow Dancing in a Burning Room together, just listening. Well, listening and sipping chardonnay.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I’m off to VS Live Las Vegas for a few days this week. I was in the UK last week and I leave early Wed-Fri for a trip this week.
If you’ve never been to VS Live, it’s a neat event. Smaller than many conferences, but it covers a bunch of different technologies for developers. Databases are a small part, and I’m always honored when I get to speak there. I hear different perspectives and questions than I find at many data professional events.
I also learn some interesting things about software development from others. To me, this is a great way to round out my knowledge. When I was more of an attendee than a speaker, I liked going to these events that are focused on multiple technologies.
If you want a last minute trip to Vegas, come this week. If not, there are some other great events coming up this year. Think about going to one if you do a lot of development:
- VS Live Nashville – May 15-19
- VS Live MS HQ – Jul 17-21
- VS Live San Diego – Aug 7-11
- VS Live Orlando – Nov 12-17
I won’t be at those, sadly, but my schedule is a little crazy in 2023, so I’m limiting the submissions. Hopefully I’ll do a few more of these next year.
Many organizations have been trying to find better ways to build and deploy software for their customers. Whether they deal with the general public or internal customers, we know that delivering software that customers use can be a competitive advantage. That’s the goal of DevOps.
While most developers and management want to do this, they sometimes forget what the goal is. Instead, they want to continue to work in a similar manner themselves while giving lip service to actual change. They often do this while pushing others to somehow produce more and better software inside the same system. I see this over and over inside various companies.
To become better, many of us use metrics and measurement of data to help guide us in determining how to move forward. In the area of software, there are a number of research reports showing which metrics are indicative of organizations that do a good job of delivering value to their customers. There are four main metrics: deploy frequency, lead time, change fail percentage, and mean time to repair. These are highlighted, though there are plenty of other things to track in your software process.
However, aiming to just improve their metrics as the primary goal isn’t going to make your software better. The goal is to deliver software that meets your customers’ needs. Quicker, better quality, more features, and all those things that customers use are what is important. These metrics are there to help guide you, not to be the targets of efforts. There’s a good article that talks about some of the downsides of just trying to improve these metrics.
The goal is the continuous delivery of value to customers. The way we do this is by experimenting with code, getting rapid feedback from customers, adjusting and improving the code, and repeating the process, learning from our efforts. We drive automation to make this smooth and easy while enabling us to get our software to customers at the pace that suits our situation. It sounds vague and amorphous, and it is.
There is a bit of an art to developing a process that efficiently builds software. It depends highly on the people involved, and on two other things. First, guiding them to improve their process and skills with references to practices that have worked well. Second, giving them the freedom and support to experiment and learn from their efforts. In doing those two things, it’s important to remember that while you can measure how well things are changing, aiming to improve the measurements often doesn’t help you improve the goal: building better software.
It’s good to measure things, but keep in mind that the measures are not the goal.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to get outside and notice how the weather feels.
This was a good tip to see before my trip last week to the UK. I flew over Mon, arriving Tue in London. I tried to get outside a bunch last week in these ways:
I spent a day at the T4 Hilton at LHR to recover from travel. While there, I went for a walk along the road circling the airport. I managed to get a look at an airplane coming in to land. It was a nice, sunny day, about 50F/10C, not too windy, refreshing and brisk.
Walking from the conference center to the hotel is a short outside section for me. Lots of rain in Wales, but I managed to take a few minutes and loop around the building during dryer moments and enjoy the very green open space and cool spring weather.
Not as much outside time as I would have liked, but it was Wales. Lots of rain during the week.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
It’s been a good week at SQL Bits. This is my favorite conference, and the organizers go out of their way to make it a fun and sustainable event.
I brought three different shirts. Day one:
Day 2:
and Day 3:
The conference center in Newport is fairly large, and dedicated to Wales. We see the dragon out front.
I even got a chance to stand next to this, and it’s pretty big.
And castles inside. The theme for the week was dungeons and dragons, and Redgate joined in with the castle theme.
Overall, the Expo was a nice success, and often busy with the food (and coffee) available inside.
We even had a mini dragon for Lego Steve
And more full sized dragons
There were the bean bag chairs everywhere, with the updated logo on one side. Sponsor logos were on the reverse side.
There was a big wall where artists were sketch noting after sessions. I thought this was pretty cool. I forgot to check after mine, but I was saying good bye and getting ready to leave and forgot to look.
Overall, it was a great week, and again, an amazing conference put on by the SQL Bits crew. I enjoyed it and look forward to coming back in the future. Wales was a great site, and I had a lot of smiles, laughs, and fun with others.
The Redgate crew was great, and it was nice to spend time with them, something I don’t always get to do at events.
The party was a lot of fun as well, with some amazing costumes. Some random shots here:
As I walked back through Heathrow, I always pass this iconic image, which both makes me smile and brings up a little sadness as I leave the UK. Another great trip though.
Thanks to everyone that came to my talk at SQL Bits.
Powerpoint here for download.
If you have questions, reach out.
Today’s coping tip is to eat mindfully. Appreciate the taste, texture, and smell of your food.
I am not a big foodie. I’ve been to some cool dinners with Brent Ozar, but I mostly go for the company. I’d be just as happy if we had the chance to eat at Ted’s or even an Applebees.
In any case, I am trying to enjoy food more, mostly as a way to build better eating habits and cope with getting older. I don’t think I have great taste, but I also am working to experiment and try new things.
A few noticeable tasty things lately:
- My wife and I went out for sushi. I ordered a new type of roll, as did she. We shared. Hers was wrapped in ginger and had mango inside. Salmon as well, of which I’m not a huge fan, but it was a pleasant mix of sweet and salty, crunchy and soft. Mine was tune with crab mix, and a firm avocado and spicy mayo. More squishy.
- I went to Hard 8 BBQ in Dallas and had the softest, tenderest, cut with your fork brisket. It was really good.
- In St Louis, I tried a number of different BBQ sauces. I tend towards spicy, vinegary ones, but tried the sweet ones and appreciated the mix of honey, sugar, and spices in there.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to take three calm breaths at regular intervals during the day.
I had to travel recently, crossing time zones and moving between planes, trains, and automobiles. I set a few reminders on my phone to do this throughout the day. Take a few minutes to calm myself and refresh myself.
I even did this a few times on my own. A few deep breaths are a great way to reset yourself physically and mentally.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to slow down if you find yourself rushing too often.
The modern world encourages rushing around. We try to do so many things, get to places, experience things, take our kids somewhere, etc. One of the things about the pandemic closing the world down was that things slowed.
Now I am back in a busy routine, and this year has felt especially busy. I’m trying to appreciate downtime and not make plans or even try to “fit something else” in. My life is busy enough, so a couple things I’ve tried.
First, if I don’t need to get something done today, I try not to do it or worry about it. I’m working to avoid “crossing something off my list” more than necessary.
If I can, I’m sitting down and relaxing more. With my wife, with a guitar, a book, or just hanging out. Spending more time with the family at home is a nice way to slow my life.
Even when working on things around the ranch, I’m putting less things on my list for a Sat or Sun, to ensure I have more a mental break from the busy work life that fills my weeks.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
It’s time for T-SQL Tuesday again, and this time there is an interesting invitation. Damien Jones reached out and wanted to host, so he gets to pick the topic this month. His invite is on the Microsoft OpenAI partnership.
T-SQL Tuesday is a great way to keep the community engaged with a monthly topic hosted by someone new each month. I manage the party, and if you want to host, let me know. If you want to find a new topic for your blog, write each month and link to the invite.
OpenAIIf you read the invite, OpenAI is an artificial intelligence (AI) research company. They’re behind the and DALL-E 2 and ChatGPT services. I’ve only lightly used these, but they have been in the news quite a bit in the last year.
Microsoft supports the company with investment and provides the cloud services they use. The invite this month is asking what we’d like to see from this partnership. What AI things should Microsoft try to integrate in Azure, O365, SQL Server, etc.
My Wish ListWhat I would love to see from this sort of partnership is more of a personal assistant AI that helps me with typing/coding/work on my laptop/desktop. I’ve seen great strides in autocorrect on my phone, and Google has some nice autocomplete stuff in Gmail and other apps. I see some of this from the laptop, but what I really would like is a better assistant that does stuff for me.
I’m thinking of these types of things:
- autocomplete, which I see is VS and Copilot, but more, smarter complete for the code I write. Recognize the types of mistakes I make.
- In the Command Prompt, when I type “get status”, recognize I’ve erased and corrected this to “git status”.
- Lately in every place I’ve needed it, I type “Floriday” for some reason. Please just fix this.
- Help SQL Prompt be smarter, and auto complete the similar types of queries I run.
- Recognize that I often write certain things over and over, bring up a template of sort, or remind me that it’s time to handle this task.
- Recognize repeating actions and offer to write some code. I have a few import tasks where I have to download a file, run code to import it, and then run a SQL script. Offer to do that for me. That’s where another human would recognize a repeating task and handle it. In this case, it’s not easy to log in and download a file from a script, but an AI assistant could integrate some semi-manual tasks like that.
- Handle some recurring search/replace stuff for me, maybe with a quick popup. In Evernote, there is some HTML added to notes that I need to remove constantly. Look for repetitive actions in an app and offer to handle them. It’s not a lot of keystrokes, but it’s also something hard for me to automate, but perhaps easy for an AI.
These are selfish, quick things I’m thinking of, but beyond what a basic voice assistant can do today. ChatGPT seems to be moving in the direction where it might actually be able to be a better assistant, if integrated into the OS and also customized and adapting to each user.
I’m sure we all repeat lots of actions, perhaps an AI might help us better handle these things. With notifications, reports of actions, and auditing, it might be something I’d trust.
Today’s coping tip is to notice how you speak to yourself and choose to use kind words.
I was downtown recently for a few days with my wife, staying in a hotel. We were busy, and we ended up eating our meals out, on the go, and sometimes in the convention center. We weren’t on a good schedule, and went too long without food at times. No gym time and a few too many drinks overall.
I started to chastise myself a few times for the diet and habits on the go, but I stopped and gave myself a break. Instead, I reminded myself to accept that taking care of myself was hard, this was only a couple days, and I should enjoy the time with my wife and friends.
Being kind to myself helped me cope with the change in routine.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I may try to blog, but I know this is a busy week. I’m leaving today for England and SQL Bits. Actually I fly to London and then spend a day there before training across to Wales with colleagues.
By my CV, this is my seventh trip to SQL Bits, and I’m excited to go back. It is my favorite conference, and I love that I can see friends from all over the world. The Summit in Seattle and MVP Summit also get worldwide attendees, but Bits gets a neat mix of many speakers and attendees I don’t see elsewhere.
I’ll try to post some pix and notes from the event, though I know it will be a busy week.
Today’s coping tip is to start today by appreciating your body and that you’re alive.
I greatly appreciate my body today. I had a long weekend, with a lot of standing while coaching and everything felt tired and ache-y each morning. However, I’m getting moving with some gym time, with some better food, and I’m grateful that I can still do most of the things I want to do.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Since I’ve had my Tesla Model Y, there have been a few things that changed in the UI. Some good, some bad, which is what I think about most software changes. This post talks about a few of these.
This is part of a series that covers my experience with a Tesla Model Y.
Side ViewThere are pretty near cameras that you can use in the Tesla. Before I got mine, I saw some people posting how they like driving with the cameras on, as they can see behind and to the side of them
At some point after I got my car, Tesla added an option to have the side camera appear when the turn signal was activated. I got a view like this one:
I thought that was pretty cool, but when I did this, the entire left side display jumped a bit to raise the car from the bottom of the screen to a point above the camera view. A bit jarring and annoying.
A few months later, another software update fixed this. I could still get the camera view, but I could also move it around, to allow me to choose where it appears. I picked this spot:
This allows me to see the view, but not have the car in the UI jump around when I signal.
Music ControlsI love music, and I love streaming Spotify in the Tesla as I drive around. It’s not a connection to my phone, but an app that logs into my account and streams independent of my phone.
However, when I first got the Tesla, the controls for music were on the far side of the (wide) screen.
It’s not a long stretch, but I do lean forward and right slightly to reach the next or previous buttons. I know I can use the steering wheel control, but that doesn’t always come to mind.
One thing that I appreciated was moving the controls to the driver’s side of the car, which you can see below.
What’s more, they added 3 cards for trip info and tire pressure to this same place. If I swipe left, I get the odometer trip stuff.
Small changes, but these made a nice difference in how I work with the controls as I drive the car around.
Customized DockAnother change made to the UI and functionality was setting 2 levels of steering wheel heat. When I got the car, I could only turn on steering wheel heating or off. Now I have two levels of heat, which is pretty cool. I mostly use the lower level, but I can use the high level when it’s really cold.
However.
Getting to this means opening up the climate controls and then finding the steering wheel in the center. I have to press this twice to get from off->high->low. Not easy to do if I’ve started driving, and it’s distracting.
I can add this to the dock, along with other controls, which is cool. If I open the list of items, I see this:
These are all the items I can select.
Now, if I long press, I can edit, and then more controls appear.
If you look at the bottom, I’ve added the steering wheel heater, which is one of the 5 most used items for me. Now I can turn it on and off without having to open a larger menu.
A super useful item, especially for people that want to quickly control wipers. Not an issue in Colorado, but in other places, I know some people get annoyed they can’t turn these on with a button. They can, the left stalk button, but a screen button makes some people happier.
The last three years have felt like a bit of a time warp. I’ll run into someone or reach out to a friend, thinking that I haven’t talked to them in a year or two. Then I realize that it’s been 4 or 5 years because the 2020-2022 time frame felt like a year or less. A long year, but still, less than the three years that passed.
Life is mostly back to normal for me, with travel, events, vacations, coaching, work, etc. all similar to the way things were in 2019. At the same time, I’ve changed and my life has changed. Some of this is aging, some is life moving on, but some is from the pandemic.
Spending so much time remotely has me a little less patient or excited about small trips, even around town. I cook more and my family is more often at home. My adult kids enjoy spending time with my wife and me. I appreciate family and time off more, which was good. These are positives for me.
Work is similar to 2019, but also different. Redgate is moving to a new, smaller office, and the culture of most people being there and eating lunch together is gone. We won’t have our SQL Servery in the new building, so I’ll be going out with friends, or bringing some sort of meal in when I’m at the office. I don’t like that.
I still get to visit customers, and more so this year than in prior years, but there are a lot more Zoom meetings and short, quick engagements than in the past. That’s a good thing, and I find it exciting.
At the same time, we have fewer events. Last year saw 18 SQL Saturdays, and we’re on pace to exceed that this year, but we’re far below the 100 a year from 2019, even adding in Data Saturdays and other events. Every event seems smaller than in the past, which is a bit disappointing for me.
Some people still wear masks, signs from the pandemic are still posted, and I find people to both be better and worse. They are more enjoyable at times, but at other times displaying more anger, argument, and selfishness than before 2020. It’s a weird mix, and I think the pandemic has people quicker to get upset at little things while being more tolerant in general. In general, the world feels normal to me, but slightly evolved from 2019.
Is life back to normal for you? It mostly is for me, though with one big difference. I appreciate life more, the people, the trips, the time, it all means a bit more after spending a few years fairly isolated and limited in many ways.
Let us know today.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to notice three things you find beautiful in the outside world.
I haven’t been outside a lot lately. It’s been cold and I’ve been busy working, coaching, and even traveling inside a lot. Not spending much time outside, so I had to stop and take a few minutes the other day.
The things I noticed:
- After a light snow, the trees were sparkling with just a light coating of frost. Everything looked bright, clean, and fresh.
- The air, while cold, was crisp and refreshing. If you weren’t out too long. At 12F, it gets cold quickly.
- The sky was cloudy and gray, but bright in the early morning. This was at 6am and everything looked like a bright Ansel Adams print.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is set an intention to live with awareness and kindness.
One of the best things about these coping tips is that I have learned to be more deliberate. I think about things and I pay attention to the world. I also try to be polite and understanding.
My awareness is higher as I age, often because I realize how many things I don’t know and how many other ways people see the world. It’s often just easy for me to get along with others by being polite.
I also think kindness is far too often lost when we get stressed, frustrated, we’re online, or we’re thinking only of ourselves. I’m working to think of others more and being kind just because it’s a better way for me to live.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I missed T-SQL Tuesday last month. I got busy and distracted with some travel. For #159, Deepthi Goguri hosted. I’ve enjoyed watching her blog and grow her knowledge the last few years and I was honored to meet her in person at the 2022 Summit. Hopefully the first of more meetings.
She had two choices: SQL Server 2022 new features or New resolutions. Both are tempting. I’m actually in a SQL Server 2022 class with Bob Ward as I start this post, taking a break from overload.
Hmmm, what do I write about?
Note: You can always fill in your blog later with T-SQL Tuesday topics. Just pick one and link to it. That’s what I’m doing here.
A Neat SQL Server 2022 FeatureI’ve sometimes been excited by new features in SQL Server, sometimes not. In this case, one of the interesting things I learned from Bob Ward was on the new contained AGs in SQL Server 2022. This is a neat implementation that essentially creates a new master and msdb database for the AG that contains the server level objects needed by the AG databases. This gets logins, jobs, etc. into these extra dbs.
There might be some holes in here, but being able to have these objects automatically synched is huge. The downside is you need to add these objects while connecting to the AG databases, so I can see some admins making mistakes and connecting to the instance rather than the AG, but I’m also hoping some automation, some scripting, etc. is used to ensure this works smoothly in your environment.
Almost every time that I attend an event, I’ll end up meeting someone that has had security issues at their company. I’m always surprised how many people have had ransomware or other security problem that didn’t get well publicized. It’s not like everyone has had one, but out of 100 people, it seems there is at least one issue.
Many of us work with third-party companies for either products or services. It’s become standard to use other firms for specific things your organization needs. However, since that’s the practice, many of those firms you partner with have their own partners. After all, they’re using other companies for specialized work just like you are.
These third- and fourth-party relationships have changed our security and risk profiles for the worse. As the numbers of data breaches and security issues grow, it’s likely that someone in your partner network has had an issue, which might mean that you have an issue. This depends on what you have contracted with partners for, but it seems more and more often this is some sort of service provided, often with your data being shared with the partner. Which could mean your data is shared with their partners.
An article recently noted that the number of partners are going up and many organizations are not aware of the risk this creates for them and their customers. There are more and more third- and fourth-party partners who have suffered data breaches, and if they have shared our data, we may have liability. The weakest link in a supply chain is the problem, and many of us have lengthened our supply chains quite a bit without paying attention.
I don’t know that there are good solutions here, but I am seeing more and more companies demanding that suppliers of services prove they have strong security practices and protocols in place. It’s not perfect, but it does help us remember that security is everyone’s business, or at least everyone with whom we share our data.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is ask someone how their day is going and really listen to them.
I had the chance to do this with one of the kids I coach recently. This athlete seemed a little distracted and annoyed by things, without great body language one day. It wasn’t busy, so I asked a few questions, wanting to know about this person, what was good, bad, what went well, not well, what was bothering them.
Not a deep discussion, but one where I could listen to them, express sympathies, but not solve their issues.
I enjoy getting to know someone I care about better, even when I can’t do anything for them at the moment.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is give positive comments to as many people as possible today.
On a travel day, I made an effort here. Those are always better for me, when I see a lot of people and can brighten their days.
- Thanking the bus driver at the airport
- Greeting the cashier at my favorite restaurant kindly
- Complementing the flight attendants on their look and service
- Chatting with the Uber driver and expressing gratitude for recommendations
- Taking a few minutes with the hotel receptionist
- Being positive and thanking a waiter at a restaurant
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
One of the very common expectations from many SQL developers involves transactions. Many developers (database or application developers) think they can open a transaction, do something, open an inner transaction (nested), and then commit or rollback the inner transaction separate from the outer one.
If you’ve worked with explicit transactions and experimented with this a bit, then you know that this doesn’t work. Recently Brent Ozar wrote a post on this as he had a client think that committing the inner transaction would release locks. It doesn’t.
Knowing whether work gets committed or not is important to data integrity. We often need to ensure that multiple things happen or nothing happens. That’s key, and if we want to decide that thing A can happen without thing B, those are two transactions. In most cases, where we’d want the behavior I described at the top, these don’t need to be nested. They’re just two transactions.
Understanding how data modifications work is important, especially if you work across different platforms and you need to ensure there is some level of durability. Some platforms use different locking strategies, some limit transactions even more, and digging into the details is important.
As technical people, we know there are many ways to solve problems, and we often spend a lot of time ensuring that users of our systems have options. We would assume our users will learn and understand how the options work, which is no different that what we ought to do ourselves. Don’t assume. Ensure you know how the database will behave if you depend on it behaving a certain way.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to make plans with a friend and catch up with them.
I decided to reach out to a friend and make time to catch up. This is someone I’ve known a long time and really appreciate in my life, but I let time get away from me and don’t always make time.
I set up a lunch and we had the chance to catch up on our lives, family, etc. It went well, though a bit short as I had some meetings in the early afternoon. We did, however, set up a second lunch to meet again in a few weeks.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
For many of us, SQL Server just works. We might get some syntax errors if we mistype things, but for the most part, SQL Server runs smoothly in many environments. However, there are some common situations that do occur regularly, and I wonder if you can guess which errors often occur?
I saw a blog this week from the SQL Server Support group where they covered the top 25 errors that come in support calls. Their goal was to see if they could document and help people better solve problems themselves and reduce the support load.
Can you guess what the first error was? I’m assuming these are in descending order, but that’s not clear. In any case, the top error was #18456, which I didn’t recognize at first. Reading the documentation page shows this is the “login failed for xxx” error, which is probably my most common error. Often because I can’t type a password correctly, but also because of an inability to select the right instance or user name. There are other causes, and it’s nice to see a long list of things people can check.
The next error was 19407, which is a cluster communication error. If that’s the second most common error, then maybe clusters and AGs need a bit more resiliency or better setup guidance. Third is an OS error with NTFS, which I’ve never run into.
If you flip through the list, I wonder how many of these errors are common for you. Do they come up often? I know I’ve seen people post on 912, which is an upgrade error and very annoying. I think some of the upgrade scripts for CUs aren’t that well written and should have better error handling inside them. That would seem like an easy one to fix and reduce call volume.
There are plenty of network errors, including the “error occurred while establishing a connection” one. That one is usually is a typo from me or a misconfiguration of an instance after installation. Lots of other errors seem network or backup related, which may not be common, but those are errors that likely cause people to call Microsoft Support.
Maybe the most interesting one is 9002, log out of space. While I know lots of people might not know how to manage space, I also see lots of accidental DBAs get caught here because they set up full backups and not log backups. Their databases are small, storage is cheap, and they encounter this a year or so after they’ve set things up. To me, this is really low-hanging fruit by making it really easy to have an automatic backup process added for each database. Just add tooling to help make this easier, or create a job when a new database is created. If this isn’t needed, let it be disabled, but for those that are installing SQL Server for some COTS application, make this a easy.
A lot of these errors are ones I’d never call support for, but I can imagine others not feeling that way. Plenty of these are errors I’ve never seen, but I’m glad the documentation is more than just a description of what happens. These updated pages give some possible causes and things that the user can do. That’s something all of us would like in documentation when something goes wrong.
Steve Jones
The grade for February is also D. Details below, but just not making a lot of progress in these areas. So far I have:
- Jan – D
- Feb – D
I set goals at the beginning of the year, and I’m tracking my progress in these updates during 2022.
Reviewing GoalsThe various sections are listed below, and I’m giving a SMART grade based on what I listed. Have I gotten much done. January wasn’t too busy for me at work, but a little busy outside of work with coaching ramping up and my wife being down from work for a few weeks.
I got a bit done, but not a lot.
ReadingTwo books on the list: marketing and coaching. I haven’t picked a marketing book, but I picked Wolfpack for coaching. With one started and one not, this is an F.
Progress:
- Wolfpack – 8%
CareerMy goals:
- Set up a tracking system based on the book for my efforts with customers Track and calculate my scores to help me better approach how I work with customers. Review this with my boss and with someone in sales
Rating: B, I rated myself on a couple more calls and got feedback on a call from another person in the company. Not a ton of activity here, but I did make progress.
Community 2 speaking engagements for community (RMOUG Training Days complete. SQL Bits on my schedule) Reach out to 10 SQL Sat groups that did not run an event in 2022 and motivate them for 2023. This should be at least 2 emails to each person/group. Reach out to all 2022 organizers and ask about plans for 2023 Start coding a tool for converting schedules to HTML. Hold office hours once in Q1 Send 3 monthly SQL Saturday updates to the community
One speaking engagement complete, one more on the schedule. So far doing well here. A few submissions in for later in the year.
I have been reaching out to other groups with limited success. I have emails out to 10 groups with limited success. It does look like there will be at least 3 events in 2022 from places that didn’t run one in 2022. A B here.
I have reached out to 2022 organizers, most seem ready to do a 2023 event if they can find space.
PersonalMy goals:
- Use my Power BI report for stats Update the Power BI report based on feedback from athletes Build 6 wooden coasters – I haven’t made enough time for my hobby here, so I’m going to start small. We need more coasters (as noticed over the holidays), so I want to build 6. Same style, different.
Same rating of a C here, as I’ve updated the report a little, I continue to use it though I need to fix a few calculations. No woodwork done with a busy life and mostly continued cold temps.
Today’s coping tip is to send an encouraging note to someone who needs a boost.
I met someone at a conference years ago and kept in touch. For some reason, I had a connection with this person and we’ve continued to lightly touch base every month or so.
I don’t know that he needs a boost, but I reached out and sent a message to say hi, to wish them well, and ask to have a chat, noting that I’m hoping they’re doing well. I got a very positive reaction, so I’m sure this was a boost for them.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to make plans with a friend or loved one.
I don’t like making plans, preferring to somewhat flow with life and make decisions close to the time when I’m ready to do something. However, that isn’t always convenient.
Recently my wife and I both got a mailing for a comedy club in the area, with some special deals. We chatted and decided to book a show, getting a date night set up together. We don’t do that often enough, but we took the opportunity this time to make plans with each other.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
It’s been nearly 3 years of Daily Coping Tips here at the Voice of the DBA. I started these when the pandemic hit and the world shut down. I’ve enjoyed them and they have helped me deal with life, and more importantly, think more about life.
However, they do eat up some time, and I am planning on stopping on Mar 24, 2023. That gives me 3 years of daily workday tips, ending on that Friday.
Hopefully these have helped you, and if you miss them, feel free to flip through the past ones and see what I’ve suggested and how I’ve implemented the tips.
If you want to set up your own calendar moving forward, looking for ways to make life better, I’ve been inspired by the Action for Happiness calendar, which comes out each month. I’ve made this one of my wallpapers for the last three years.
Recently I needed to add a computed column to a table and realized that I didn’t remember the syntax. This short post show how to do this.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
Adding the ColumnI had a table, OrderHeader, and wanted to add a new column, OrderedByDate. I can do this with a simple ALTER statement and an ADD. The inserting part is the computation. You use as AS clause with the computation instead of a datatype. For example, my code was:
ALTER TABLE dbo.OrderHeader ADD OrderedBy AS OrderDate;GO
This added a copy of my OrderDate column with a new name. This is useful for zero downtime deployments in some cases, and in this case, I wanted to just have a copy. However, if I wanted some calculation, I could easily do that by specifying this as I would in a SELECT statement. For example, if I wanted the computed column to be a week later, I could do this:
ALTER TABLE dbo.OrderHeader ADD OrderedBy AS DATEADDD(DAY, 7, OrderDate);GO
This would add a week to the original value and return that. I can likewise do any sort of string or numeric manipulation I want. A common one is adding or multiplying two columns together for a new value. For example, adding various charges for a total in an order.
When you do this as a computed column that is not persisted, no space is taken in the actual table rows. If you persist this, then space is used.
More information on Microsoft Learn.
SQL New BloggerI realized that I needed to double check the syntax in the docs, and when I did, I took this as an opportunity to write a short blog post. I grabbed a link, wrote some code, and then spent 10 minues knocking out this post.
If some employer does this a lot, they might search your blog to see if you can do this. A nice few posts on how to do this, what persisted does, how you index this, etc. Take a few minutes and start blogging on topics like this throughout your week.
Security has become better and better in many organizations. At the same time, hackers and malicious actors are doing a better and better job of finding new ways to attack systems. Some work to target specific individuals, often because of government or industrial espionage. Most of us aren’t likely to deal with those issues, unless we work with (or are) someone that is very important in a particular situation.
Instead, many of us deal with wider spread attacks that look to exploit vulnerabilities in technology or humans at scale, hoping to find the weak links. Lots of people I know have dealt with viruses in the past that shut down systems, and more recently, had to rebuild systems crippled by ransomware. Despite their best efforts, this often means lots of extra unexpected work, combined with the stress of falling behind on our commitments. We have plenty of other work to do.
Windows has been vulnerable throughout its history, and it appears, lately through a data problem. There is a class of attacks that look to use approved, though old and unpatched, drivers as a vehicle for gaining a foothold inside a network. This was a problem (called the BYOVD issue) and Microsoft addressed this with a block list that was used to prevent the loading of vulnerable drivers. They updated this list through Windows Update.
Except they didn’t. In database terms, we had an eventually consistent set of data, which was being updated at Microsoft, but not being sent to client workstations. There were over 3 years of the list not being updated, despite assurances from Microsoft that Windows 10 PCs were protected. There are instructions for manually updating your machine.
I don’t envy this being a process I’d want to build. Getting data from security researchers or elsewhere, putting it in a database (I hope), then exporting this into a text format and getting that loaded into the Windows Update process, all while trying to ensure the process is secure along the way. That can’t be easy inside a large company like Microsoft. At the same time, not noticing this wasn’t working isn’t excusable. Likely there were issues, but my guess is someone didn’t want to admit a failure and get a bad annual review.
Data sync issues are nothing new, and many of us struggle with these on a weekly basis. However, these are important issues. Replication can fill log files (and disks), broken ETL processes can cause execs to make poor decisions, and in the security space, not updating drivers and block lists leave us vulnerable.
This situation isn’t excusable and Microsoft ought to be ashamed. Some of these execs ought to lose bonuses, at the very least. It’s also not excusable in our orgs. We ought to be sure we’re patching on a regular basis and minimizing the attack surface area we present. It’s the least we can do as IT professionals.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to call a friend to catch up and really listen to them
Actually a friend pinged me to ask about a call recently. I made some time and set up a call.
I’m glad I did. A friend is having changes in life and wanted to talk. I feel similarly, and in this case, we had some similar challenges that we could talk about, support each other, while enjoying catching up.
Worth the 15 minutes of my life to do this.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to thank three people you feel grateful to and tell them why.
This is something I tend to do privately, thanking people who’ve impacted my life. I’ll do that, but in general, here are the people I want to thank.
Friend 1 is someone I’ve known for many years in Denver. We get together periodically for lunch or dinner, but I’m always happy to have an ear to vent about life, kids, marriage, work, etc.
Friend 2 is someone I’ve known from my work and community experience who has been a friend and support system in many ways over the years. We’ve had the chance to share some tough times and help each other, which I appreciate.
Friend 3 is a friend at work who’s been someone that has supported and helped me grow myself as an advocate.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today is the corporate Wellness Day at Redgate. It’s a day off for almost everyone in the company, unless they have something that can’t get moved. A few support people, a few others, like me, that are traveling. We, however, will get another day that we can take off.
When we revamped benefits for 2023, one of the things that we wanted to encourage was health and wellness. We added a couple days off, like this company holiday and a birthday day off. We also reduced the ability for employees to get out of vacation by not buying back days and emphasizing that managers should ensure they work with people to take their entire allowance. We can carry over 5 days, but no more.
I like the idea of this day off, as it means people take some time away and won’t come back to a bunch of emails or other things that people sent during their break. Instead, the company is essentially shutting down.
I don’t mind missing today for travel. I’ve got things to do and I’m looking forward to working with a customer tomorrow. However, I also value my days off and I am looking to try and get a day later this week or next and get up to the mountains.
Whether you think this is a good idea or not, I would hope you admit that it’s nice that the company is trying to ensure employee wellness. Last week had a number of meetings for Mental Health Awareness Week, and to cap off those efforts, we close today.
I’ve never had a corporate closure like this as a benefit. I’m looking forward to next year and taking the day off with everyone else.
I hope.
The short answer is no.
The longer answer is it depends, and perhaps to be more complete, ChatGPT likely will help us produce the simple, tedious queries with much less effort.
There has been a lot of news about AI and ChatGPT and how well it performs for a computer. That last phrase is important because while it’s impressive, I don’t know that any of us would consider ChatGPT to be in our list of the Top 5 Dream Dinner Guests. The tool isn’t that impressive compared to most people, especially our friends.
I look at Ayende Rahien’s blog often, and he recently had some thoughts on this topic, experimenting with ChatGPT and coding. Some things worked; some didn’t. In general, to get good code, you still need a subject matter expert, which is also the case with humans. We have plenty of people writing code that aren’t great at their jobs. We do see a lot of bad code, but we also have some great coders that help others to learn or just refactor their code later.
So is ChatGPT better than a below-average developer? I don’t know. While the tool will get better, one advantage with people is we can complain to them, or send them problem queries, and they’ll learn to paste better code in from the SQL Server Central forums or StackOverflow or somewhere else. Perhaps ChatGPT will learn, but will it learn to improve and refactor code? I’m not sure how easy it will be to teach the system to edit something rather than just produce new code.
I do think that the assistance features of AI, suggesting ways to complete sentences or lines of code, can be very valuable. They are a great productivity tool that can enhance your ability to get work done. If they learn to work in your style, suggesting the things you’ve done in the past, then these tools will help. As with most tools, they aren’t a replacement for your knowledge, but merely a lever to make you more efficient.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to tell a loved one about the strengths you see in them.
For one of my kids, I’m letting them know that I see:
- responsibility for meeting their commitments
- independence in making their own decisions
- accountability for their decisions
- strong willed to drive themselves to accomplish more than they have to
- compassion for others
I’m proud of all my kids, and I took time to let one of them know.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I ran across an article on EV myths, this I thought was interesting. The myths are things that I see in comments on all sorts of articles or hear from people. They are:
- EVs take forever to recharge
- EVs can’t travel far
- EVs are slow
- EVs are unreliable
- EVs are super expensive
I haven’t experienced any of these, with my EV charging fairly quickly, I go up to ski in mine, it’s fast (perhaps too fast), it always works, and it’s in line with luxury cars.
The last one gave me pause, because my Model Y was the most expensive car I’ve every bought. It actually bothered me for months in the summer of 2021 until my wife got me to sit down and think rationally about it.
The car is a $55-60k car. That is pricey, and not for everyone, but it’s in line with a petrol car from BMW, Audi, Mercedes, etc. Most of the other EVs, Polestar, Ionic, etc. are similarly priced. The basic Model 3 is in the 40s.
Pricey, but not crazy.
The high end Teslas (X and S) are like a high end 911 or BMW.
I’ve had no maintenance in my model Y in 17 months. I’ve had it always charged when I start the day, so I almost never need to charge during the day. The times we’ve had to charge during the day (our summer trip in the mountains), the time was more than gas, but not much more than a pit stop for coffee and the bathroom.
There was another article from the Atlantic, which I’ve seen reposted a few times as they are supposedly a world class, trustworthy journalism site. I think they are, but they harp on costs, which are like other cars. They can get high quickly. I agree that starting the Ford F150 around $50k but quickly going to $80 isn’t great, but it does give options to others.
From what I’ve learned in a year and a half, the vast majority of the time, 95%+, I got less than 120 miles in a day. I could likely live with a 200-220mi range car. YMMV, but it really depends on you doing a little work to understand how you drive, how often, and how the car sits. There are days we go 250miles and need to charge, but that’s rare.
Honestly, the more I drive the car, the more it’s just a car. The exception is longer trips, in which case, then my driving becomes like a trip through Wyoming or Montana with a diesel vehicle. I do a little planning.
There’s a common party question about which 5 people would you invite to a dinner party? Often this is amended to include living or dead people, and it’s often interesting to hear people tell you who they’d invite and why.
Since most of the people reading this work in technology in some way, I was wondering who you would invite to a party that’s related to technology. Living or dead, however tangential, is there a list you can come up with?
I’ll give you my list and then a few thoughts. For the list, I’ll say Steve Wozniak, Sal Khan, Reed Hastings, Gladys West, and Lawrence Lessig. Maybe not all technologists per se, but all had an influence on my life using technology.
Steve Wozniak because he was a geek hero and he seems like a fun person. Someone that enjoys technology. I had an Apple II early on and always wanted to meet him.
Salman Khan because I think his use of technology to teach others, for free, was incredible. Education is something that’s a big part of my life, and I appreciated his lessons helping my kids at times. I think someone that sees the world in this way would be a great conversationalist.
Reed Hastings has been an incredible businessman in the tech world. He founded Pure Software and Netflix, and has been on the boards of Microsoft and Facebook. He has had an influence on education efforts in California. I would love to listen to him talk about business and technology.
Gladys West is a mathematician who worked on a number of topics that built computer models for submarines and analyzing satellite data. She has had quite an influence on the GPS system many of us use every day in our phones. I’d love to know what it was like for someone like her to work and grow in our field.
Not least, just last. Lawrence Lessig is a professor of law, and someone that has worked hard to ensure our legal world better represents fairness and ethics in technology. I’ve read a number of his books and would be interested in talking about how we ought to view the digital world from a fair rights perspective.
That’s my list. What’s yours?
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to be gentle with someone you feel inclined to criticize.
It’s very easy, and maybe very human, to start to criticize others for doing something we wouldn’t do, or at least, something we don’t think we’d do. I sometimes wonder if we are much more likely to do this with people close to us than those we know less well.
My kids are adults now, and they are in charge of their own lives. They make mistakes, as I do, and there are times that I want to point out they could do better, or take other advice, or something else.
I am learning not to do that. One of my kids did something that cost them more money than they expected. They were upset, and it would have been easy for me to criticize the action. Instead, I have been working on being sympathetic, listening, and asking what they think or what they do. Let them work through it without me trying to solve the problem, take responsibility, or criticize.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to give sincere compliments today to people.
A tough day recently coaching, but I kept this in mind. Complementing parents, competitors and fellow coaches, and even my kids. All while not having things go our way.
It was tough, but I was working to find something good to say about a variety of people.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I’ll be at VS Live in Las Vegas this March to discuss zero downtime deployments. If you want to come and join me for this session, or any of the other great ones, register today and save $500 with the promo code “Jones”. You can use this link to register.
To simulate the effects of deployments on a workload, I built a small client. It’s nothing great, and likely some of you will laugh at my C# skills, but it works well enough. It’s a simple Windows form application that writes to a text box. However, it’s valuable to determine if there are any issues when you’ve made a deployment.
This post looks at the rough design of the client. Code is available in this repo: https://github.com/way0utwest/ZeroDowntime
Using WPFI wrote a small app a few years ago to test and present on Always Encrypted. This was a basic WPF app that added the proper values to the connection string for Always Encrypted and let you query encrypted data (or not).
Like all mediocre developers, I copied and pasted that project into a new folder and set about modifying it. In this case, I set up a loop that continues to run and execute some database code, essentially using this loop:
while (iRunQuery > 0)
{
I set this value to 0 initially, and when a button is clicked, it’s 1. This then runs a bunch of lines to decide which DB code to run. I’ve mostly made this stored procedures to make it easier to adjust demos without touching C# code.
It’s not pretty.
At the bottom, I have this (outside the loop)
```
Application.DoEvents();
System.Threading.Thread.Sleep(100);
``` This is designed to catch me clicking a “stop” button that will set the variable back to 0. I added the delay because otherwise this runs a bit fast.
I have a few option buttons that adjust what code I’m calling, so I can simulate toggling feature flags on and off. I also log results to a window so you can see them, and I catch errors and log those. Errors are also counted, so we can see the impact of “non zero-downtime” changes.
It’s not a great example of software, but it does work.
In the past, many businesses hired employees whose role was deciding which prices to charge for their goods or services. At one point, organizations largely set prices based on their costs, though over time they tend to look at their competitors and set similar prices. If, however, management from multiple companies went into a room and determined prices, this would be price-fixing.
Price fixing is illegal. In many countries, we would not allow different companies to work together in a way that might reduce competition or take advantage of consumers. However, that might be a struggle in the future as we find companies using various services to help them manage their systems.
In this case, a few different Las Vegas hotels used the same company to help them decide how to price their rooms. Rainmaker is a revenue management platform, which uses lots of data to help hotels price their rooms in a way that maximizes their revenue. That sounds great, but if this company is successful and many of their clients are in the same location, this is really a way of sharing data by proxy. The hotels are being sued because of their use of this platform.
This one of the problems (or advantages) of lots of data. It allows information to be drawn out of data that wouldn’t otherwise be obvious. Certainly, lots of companies look at their competitors and make decisions based on what they see. I’m certain there are lots of people inside airlines constantly checking the prices of their competition. However, they are gathering this data independently and making their own decisions. If Rainmaker were used by American, United, and Delta to set prices for flights, I imagine many would see this as an anti-trust violation.
Big data is powerful. It can help give an organization an advantage over its competition. This is one reason lots of companies hire data professionals like us; they see data as a very valuable asset. However, in this case, I feel that one company selling this data, or rather the conclusions, to competitors is a problem.
I expect more problems like this in the future as smart people look to harness the power of data and sell their services to competitive companies in many industries.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to share something you find inspiring, helpful, or amusing.
Maybe not inspiring to you, but it was for me. We recorded a number of customers and partners at the PASS Summit, asking them what they thought about our work at Redgate.
There’s a cool playlist that is inspiring to me. Makes me want to work harder and proud of our work.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to respond kindly to everyone you talk to today.
This is often easy for me. I did this last week while traveling.
I got coffee on the way to the airport. I greeted the cashier with a good morning and thanked him. I did the same at the airport when I grabbed breakfast, but also asked the young lady how her day was going.
On the airplane, I greeted the flight attendants while walking in and when they served drinks with a hi and thank you. Same when leaving. I also thanked someone that paused to let me walk ahead of them.
Asked the Uber driver how he was and I appreciated him picking me up.
Had dinner and pleasantly chatted with the bartender.
A happy day.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
It’s President’s Day in the US today, which is a holiday for me. I don’t think of it as much of a celebration, but it is a day off. So, I’m taking the day off and republishing some content.
This is a US holiday, but it’s Louis Riel Day in Canada, and celebrated in quite a few other countries as well for different reasons. I hope quite a few of you get the day off and don’t read this until tomorrow, Feb 21.
In any case, enjoy the time off when you get it. Easy to get caught up in work and overwork yourself. I am working today, but in coaching, not technology.
Today’s coping tip is to appreciate the good qualities of someone in your life.
We have a person in Redgate that does a lot of work to support our sales process. This individual has been a force for producing resources that many people, including myself use.
I don’t want to name them publicly, but I really appreciate the work ethic, the flexibility, the willingness to learn, to admin mistakes, to go the extra mile, the friendliness, and the enthusiasm.
There’s more, but these are the top ones I appreciate.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to support a local business with a positive online review or friendly message.
When I discover a business I like or one that delivers value or joy, I have been trying to leave a review or message online. This is how many people find new businesses, and the local ones can always use support.
Santortas is a local restaurant near me that I love. Their breakfast burritos are amazing. My only complaint is they don’t open until 9, so it’s mostly a weekend treat for me. I wish I could get one at 7 when I go skiing. I left them a review on Google.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I was driving home from the mountains recently and witnessed an accident. I have proof.
This is part of a series that covers my experience with a Tesla Model Y.
The DashcamThe Tesla Model Y (and all models) has a number of cameras built into the system. This is recording all the time, and if you use the USB they give you (or buy another), you have the clips saved for the future. I bought a USB drive, and it’s been in the car recording when I drive. I sometimes use it when parking, but often not since that drains the battery as the car can’t sleep.
While driving home, I was slowing as I came to a red light. I saw two cars cross in front, getting ready to turn left in front of me. One accelerated, right in front of a car coming from my left.
The car going South, from my left, likely didn’t see the turning car as there were two cars stacked up to turn. The result is below, with a collision.
I haven’t been contacted by the police or insurance as of yet, but if they call, I’ve got video.
I like that this is built in. With modern cars having all sorts of computer tech in there, and cameras relatively cheap, I’d really expect that more manufacturers would put these in their offerings in the next few years. Or insurance companies would ask for them.
One annoying thing is the UI in the Tesla for viewing these is really slow to read the SSD and pull them up. Grabbing this out of the car and plugging into my computer let me pull off the video in a minute or two.
The hard part then is remembering to put the SSD back into the car
Many of us work inside an organization that has a process for building and deploying software. We may find our org doing this well, or we may feel our process is poor with lots of room for improvement.
A lot of the discussion around how to be better at building software in the last ten years has been around the philosophy of DevOps. This concept doesn’t really prescribe how to build software, but give you goals to aim for. That means you still need to take the ideas of flow, feedback, and learning and decide how you implement them with your staff. What practices do you follow to ensure you can deploy quickly anytime your code is done? These can include ensuring you’ve tested code, getting feedback from customers, and more.
I ran across a post on recommended software engineering practices for an organization. The list includes seven things you should do:
- Keep documentation in the code repo
- Have a mechanism for test data creation
- Use rock solid database migrations
- Create templates for new projects
- Automate code formatting
- Automate a process for new dev environments
- Automate preview environments
This is a set of things I often preach to customers as well, especially 2, 3, 6, and 7. I often focus on the database and having curated test data, migrations you can count on and easy setup is important. And, of course, with SQL Prompt, you don’t need formatting ;). Just kidding, that’s important, too.
These are solid practices, and none of them are that hard to set up, but they do require some discipline and willingness to work as a team and maintain your process across time. Each of these items needs some care and feeding across time to remain relevant and helpful to your staff.
Do you have good software engineering practices? Are you proud of them and would you bring them to a new position? Or perhaps you wish your team would adopt better habits and a different mentality towards building software.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to focus on being kind rather than being right.
This is one of the things that grows my wisdom over time. I don’t need to be right anymore, at least not often. I can try to guide and assist without winning over colleagues, friends, family, etc.
One of my kids was having a bad time, and it was because of a poor decision on their part. Rather than try to make sure they knew the problem, or that I knew it, I tried to support and listen.
It is helpful for me to remember that I’m really responsible for myself, and somewhat my wife, but no one else. That includes my adult kids.
This is a good coping skill that reduces stress in my life, when I can support, offer, assist, but not take ownership or become invested in another’s decision or action.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Basecamp (formerly 37 Signals) is quitting the cloud. One of the founders gives some reasons, and he had some detail in a tweet on what they’ve spent in the cloud the last few years. Over USD$3mm on various services, though their costs in search seem very high. I don’t, and haven’t, run as busy a business as they do, so I don’t know if they’ve truly done a good job architecting things and setting up services. They say they have, though I’d expect everyone to say and think that.
However, I’ll assume they are correct and they can’t optimize things any more than they are currently. Their decision makes some sense, and I agree with it. I’ve been surprised at the growth of the cloud, both in size and how quickly people are moving to the cloud. I’ve also been saying for years that if you have a steady or known workload, the cloud is likely very expensive.
Maybe that’s worth it for your organization. Not dealing with physical resources, maybe having slightly less staff, maybe less CapEx vs. OpEx. Those are decisions for management and finance people. For most of us, the cloud both simplifies some tasks and makes others more complex. Provisioning, testing out Proof-of-concepts, and scaling are easy. Identity protocols, gaining (and keeping) knowledge of how various options work (networking, storage, etc.) , and keeping track of resources become more complex. Not to mention the world constantly shifting under your feet as cloud providers change how their platforms work.
There are costs in both hard dollars (or your currency of choice) and in the time your staff spends dealing with a new way of doing business. The calculation of whether this is a cost that makes sense is very dependent on your situation. I have customers that love the cloud and others that hate it. The value they get varies dramatically and some would never go back to data centers while others are ready to work long hours to leave the cloud. Overall the sentiment is the cloud is great, but like many decisions made by management there are particulars that baffle the technical staff.
The one thing I have learned about the cloud is that it takes a different sort of mentality from staff than on-premises resources. We have to learn to spin things up and down, scaling as needed. We need to better understand budgets and not look at costs as though they were personal expenses. We also need to be flexible with resources, understanding that machines that are idle are not sunk costs; they are ongoing costs.
The cloud is amazing, and I think it is very useful in lots of situations, but a blanket move to the cloud can be expensive. Make sure that everyone involved in moving to the cloud understands that.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to look for good in others, particularly when you feel frustrated.
We all deal with conflicts with others throughout out lives. How we handle those often says a lot about us. As I age, I try to avoid getting emotional and upset, reacting too quickly, and working to understand why someone said/acted/did something.
Today, I’m thinking about my dog. He’s a frustrating beast at times, growing out of puppy stage, but still young. He was a rescue/adoption and a nervous boy. He loves my wife, but he bonds well with all of us.
One of our struggles is leaving him at home. He mostly goes to work with my wife, but there are times where he’s at home and I or someone else needs to leave and he wants to follow their car. I was heading to the gym recently, and I had to turn around a couple times to put him back in the house. We don’t want to lock him in as he’s still getting housebroken, so we’ve been giving him a treat to keep him busy while we leave.
A frustrating period of time as I was trying to get gym time during a lunch break, but I had to remember how much this guy loves me and how often he makes me smile. He’s not trying to be frustrating. He just wants to be with me.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to thank someone and tell them how they made a difference for you.
I’ve had a lot of success in my life, as well as a lot of help along the way. I’ve worked hard, but by no mean has that meant I did things by myself. I think far too many people forget about the support they’ve had from others.
One person that made a difference in my life was a coach that my daughter had. He was a great coach for her life, but he inspired me as well. Like me, he had a daughter that played and then he started coaching. He wasn’t more than a casual volleyball player, but learned to be a really good coach and a wonderful human.
He taught me a lot, answering my questions, giving me guidance, and he was one of the people that got me thinking about coaching and then moving into that role. I let him know recently how much he changed my life.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
In my previous post, I set up the Flyway Desktop projects for SQL Server and PostgreSQL. I also added a table to each platform for development. In this post, I’ll look at how I let Flyway know what already exists in my system with a baseline.
This is part of a series of working through Flyway and Flyway desktop to demo database changes. Disclosure: I work for Redgate Software
Baseline v Flyway BaselineI find these terms to be slightly confusing, especially when I look at Flyway vs. Flyway Desktop. In Flyway, there is a “baseline” verb, which you can run at the CLI. This will mark the state of your database at a level and adds the flyway_schema_history table to the schema in which you are working. This baseline causes Flyway to ignore all migrations up to the baseline level.
In my testing, when I run this on my database, it defaults to V1 for the first migration script. That’s usually fine, but as I wrote, this can cause issues.
There is also the concept of a baseline migration, which is a Bxx script, and this contains the definitions of all the objects that already exist in your target databases. This ensures that as FW and FWD track and deploy changes, they don’t try to redeploy those migrations that are at a level lower than the baseline numbering (the xx).
Creating a BaselineMy development database for SQL Server looks like this:
There is an object in here, but it’s not in any other environment. Both Integration and QA (and the others) have no objects.
In this case, I don’t need a baseline script, because I want this table to deploy to the downstream databases.
I do, however, need a baseline. I need the baseline marker in my databases to note that we have a base version. This will give me a starting point, but also ensure that FWD creates migrations that are numbered higher than my baseline.
I’ll add this in two ways. One with Flyway Desktop and one with the Flyway CLI.
The Flyway BaselineFor SQL Server, I don’t need to worry about any objects in downstream databases, so I’m just going to run the Flyway CLI. From a command line, I’ll run this code:
flyway baseline -url="jdbc:sqlserver://localhost;instanceName=SQL2022;databaseName=FWPoC\_1\_Dev;encrypt=true;integratedSecurity=true;trustServerCertificate=true"
This is run from my project location, though I’m passing in the connection string from Flyway Desktop as I don’t have a flyway.conf file configured for this project. Things work from the FWD gui, but not the CLI.
This works, and I see these results. Note the flyway schema history table is created at the bottom, and the version of the database is set to 1.
Now when I run Flyway info with that URL, I get this. There is an entry in the version tracking for this table:
I can also see this table in my Object Explorer:
Flyway Desktop and PostgreSQLI’m going to use FWD for my PostgreSQL project. This will do some of the work for me and give me the option for a baseline script.
Note: I set up a shadow database first.
I click “Create baseline” and this asks me for a target. After all, I’m trying to ensure I don’t deploy anything to prod that’s already there.
When I click Add target database, I get a connection dialog. I fill this in with the credentials for prod. This returns me to this screen below, where I see my prod database, which is at this port with this name.
I click Baseline and it goes to work. There’s nothing there, so this returns back to the blank, Generate Migrations tab.
However, there is no baseline or schema tracking table. I didn’t have a poc schema, so perhaps that’s an issue, but that’s OK. We can fix this.
In the Migrations tab, I see this:
That configures this tab to look at (and work with) this database.
In general, I know we won’t be able to see production, but this is a PoC. However, this is something that I, in general, don’t want to do. I want to work with dev/test environments, so let’s do that.
I’ll configure my QA environment. I click “configure target database” and I get this screen. These are all the databases for my project. Here I’m going to click “delete” for production and then I’m going to click the Add and configure my QA database. Once I do that, I’ll see this:
Baseline added for PostgreSQL.
Success.
I saw this tweet recently, where Richie Rump asked what has changed in T-SQL since the SQL Server 2012 version. A few people from Microsoft responded that there were changes in all versions, and while I think some versions have few changes, I decided to look.
SQL Server 2012 introduced the window functions with the OVER() clause to SQL Server. This was a huge change in that many aggregate queries were much easier to write without needing complex GROUP BY lists and subqueries or unions to join together different data. While I’m not an expert by any means, I find lots of queries for reporting easier to write with the window functions, and I’ve grown to enjoy using these in code.
Looking across other versions, I’ve seen these changes:
SQL Server 2014
- UTF-8 for Bulk insert
- SELECT..INTO works in parallel
- In-Memory OLTP language enhancements
SQL Server 2016
- temporal tables
- JSON support
- more In-Memory T-SQL changes
- Security – DDM, RLS, AE T-SQL changes
- R services
SQL Server 2017
- graph query
- CONCAT_WS, TRANSLATE, TRIM, WITHIN GROUP
- BULK INSERT options
- Memory-optimized enhancements (CASE, TOP, JSON, computed columns
- Python language services
SQL Server 2019
- Graph enhancements
- UTF-8
- Java and other language enhancements
Some of these were to support other features, so perhaps these aren’t really T-SQL changes per se. If I look at PostgreSQL release notes, I see enhancements and changes, but relatively few new language changes. Certainly, there are some additions, but lots of improvements, which I think reflect the nature of a mature product. Not a lot of new things, but regular improvements and refinements to existing items.
I’ve been working with SQL Server since 1991, and it feels like T-SQL has grown a lot in that time. Back then it felt like there were relatively few keywords and functions, requiring complex coding for tough problems. Now, with the way the language changed a lot in 2005, 2012, and 2016, it feels like we have a lot of tools at our disposal. We could always use more, and we got some neat ones in SQL Server 2022. I hope to see more useful changes in future versions to come.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to show an active interest by asking questions when talking to others.
I listen more and more these days. I’m trying to input less and hear more, while still being engaged.
However, I know that sometimes I need to carry my weight in a conversation and ensure people know I do care. I want to ask questions, but not those that put them on the defensive or imply judgment. Instead, I want curious questions.
While talking with a kid I coach about their weekend, about which they were very excited, I showed them enthusiasm, but I wanted to keep hearing about what this kid liked. So I asked about the best part, the parts they’re repeat, the reactions from others. Little questions to let them know I was listening and caring.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to send a message to let someone know you’re thinking of them.
I reached out a friend the other day, someone I know from the SQL Community, but I haven’t talked to for some time. Something made me think of them, so I just sent a note to see how they were doing and let them know.
Didn’t need anything, or want anything, just wanted to touch base. It was a nice feeling I felt from doing so.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to write down your hopes or plans for the future.
Hopes are vague, and plans are easier, so here are some plans:
- Plan a trip to South America or Japan for 2023
- Go back to Italy with my wife this year
- Move back to coaching younger kids
- Rebuild a generator shed around the ranch
- Learn to patch concrete
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
One of the challenges for me when working with customers is getting them to think about how to change their software process. Often they want to solve their problems, but no one wants to alter the way they work. Whether that’s the protocol for capturing code or managing servers, it seems that changing the way we work is hard for many people. They want everyone else to change. Or they want a magic tool that solves problems without them changing the way they work.
Unfortunately, I’m not a magician.
This is a common problem in many organizations. We want to be more efficient and effective, but actually changing our habits and culture is hard. Management should lead the charge, and they want to, but they can struggle with how to do this. Often they focus on changing technology, and not actually improving the way their company works.
There’s an interesting article on digital transformations and whether the efforts are worth the investment. In many companies, someone makes a good argument for a course of action or a project and then drives it. Others participate, but often a person pushes this forward, usually because they have some stake in the outcome. A bonus, a reputation, or just pride, whatever matters to them becomes the reason for continuing, even if there isn’t enough value from the effort. This can be because of an institutional culture that wants to finish projects, wants to find some success in a course of action. Whether a human or an organization, pride and inertia often keep us moving forward, often without any other support or analysis of how well things are progressing.
Effective dashboards enable everyone to see current status and progress, and to make better course corrections, helping to move from a command-and-control model to a coach-and-communication orientation. Many organizations have adopted KPIs, dashboards, and other ways to analyze parts of the business, but this isn’t always something we do well in software development. We tend to look at metrics that management cares about, and on which we are measured, rather than metrics that might help us improve how we work.
Part of digitally transforming a business is also transforming how technology is used. Part of that is us, as technologists, learning to be better and more effective. Whether this is in development or operations, we can often improve how we function. We need coaching, and in many orgs, that coaching has to come from within, from the people in a team asking others to do better. And to allow others to ask us to do better .
Coaching is often teaching from a different perspective. It’s helping someone see what they don’t see themselves. This might be new knowledge, but many times it’s just reminding the individual of something they know, but aren’t doing. As we are asked to do more, be open to coaching and be willing to help coach others. Become a role model that helps transform how your organization uses technology.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to go to bed in good time and allow yourself to recharge.
I need this reminder. Lately I’ve struggled to sleep well, through the night, and I’ve had some rough mornings. It seems this usually happens when I have early meetings, which makes things even worse.
I’m making it a point to get to bed by 10 for a few days and see if that helps me at all.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Next week I’m heading to the Lowry Conference Center in Denver for the RMOUG Training Days. The Rocky Mountain Oracle User Group has put this on for years, and I’ve been honored to speak a few times.
This time I have two sessions, one live and one virtual. The live one will be Flyway tips and tricks, sponsored by Redgate. The virtual one is my Branding Yourself for a Dream Job talk.
Register today and join me to learn about some Oracle and MySQL
Today’s coping tip is to go to bed in good time and allow yourself to recharge.
I need this reminder. Lately I’ve struggled to sleep well, through the night, and I’ve had some rough mornings. It seems this usually happens when I have early meetings, which makes things even worse.
I’m making it a point to get to bed by 10 for a few days and see if that helps me at all.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
In order to generate migrations, we need to configure Flyway to use a shadow database. This post looks at that process.
This is part of a series of working through Flyway and Flyway desktop to demo database changes. Disclosure: I work for Redgate Software.
Configuring the ShadowThis is an empty database where we run the migration scripts to verify them. Since a user might edit or create their scripts, we want to ensure there are no problems with the syntax or execution with other scripts. This is also a place where we keep the “previous state” of your development database and use this to detect the changes you’ve made.
This database gets cleaned, meaning objects get dropped, regularly, so you configure a space for this. It can be a separate database, or just a schema (more Oracle focused).
For me. I’m going to create a new database in postgreSQL to support this. As you can see below, I use the simple CREATE DATABASE syntax.
Once I do this, I go back to FWD and click the “Generate Migrations” tab. The first time I do this (and only the first time), it asks me to configure a Shadow database.
I click this and get a connection dialog, similar to what I have for my development database. In here I enter the credentials for my shadow database, which are similar to my development ones. I test the connection and verify I can connect.
That’s it. Now I’m configured for a Shadow.
The process is similar for other platforms, just with different credentials. If you need to learn more, you can read about this in the documentation.
When I first started working in technology in the 90s, it was a time of outsourcing lots of work overseas. Many large companies followed the wave of manufacturing in the 70s and 80s by many companies, including lots of semi-conductor manufacturers. I watched as a number of jobs moved overseas, though fortunately not mine.
In the early 2000s, I worked at a company as a manager, where I was involved in some of the discussions about outsourcing a lot of our IT operations to another company. It was a scary time for my friends, as the move would have left a lot of them looking for work. The CIO tried to say that many would get jobs with the outsourcing supplier, but couldn’t say how many.
I commented in a meeting that it would have to be less than 100% of people for there to be any profit. That statement ensured I needed to look for a new position, which I happily did. The company didn’t outsource at that point, but being involved in the discussions helped me realize how different management and workers often view the business of IT.
There was an article on when to outsource, which talks about the benefits and drawbacks. For most of the benefits, the reasons why you do this are very similar to why you might move infrastructure to the cloud. There is speed and flexibility, as well as simpler internal operations, when another organization handles those functions. The downsides are that your organization needs to manage another, and you might not have the control you desire.
The same thing you could say about the cloud.
The costs could be lower, with outsourcing or the cloud, if you can reduce unnecessary resources. That can be hard to do internally, especially as you hire staff. Each additional person might be necessary in the short term, but if when they are no longer producing a positive ROI, it can be hard to get rid of them. Consequently, with an outsourced company, often contracts dictate the staffing levels, and those contracts aren’t amended or renegotiated often. That could leave you with lacking services you need or paying for those you don’t.
I don’t know that it’s easy to decide when to outsource. My view has often been to keep some level of in-house staff, but augment them with some sort of managed service provider that can provide additional resources when you get busy. Whether this is a DBA service, like https://dallasdbas.com/, or a development effort, like Crafting Bytes, adding staff in limited quantities is often the best way to move forward.
These smaller groups also provide opportunities for some of you that might want to find a different type of employment arrangement. Usually more flexibility, a warm, friendly atmosphere, and the chance to grow with a small business.
Outsourcing might sound like a bad idea to many corporate employees, but it does provide opportunities for you. Keep working on your skills, network with others, showcase some knowledge with a blog/article/speaking slot, and you might hedge your bets in the event your employer makes a decision about outsourcing that doesn’t work for you.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to get back in contact with an old friend.
This is something I don’t do too often, but I have done a few times. I’ve met a lot of great people in my life. Some I worked with, some I encountered in other parts of life. When I go to the UK, I usually see if I can reach out and get the chance to meet someone I know from there for a meal or drink. Which reminds me, I need to reach out to someone and try to make plans for June.
Around here, I’ve met some really interesting people as a part of coaching. I took time recently to go watch a few kids play that I coached in the past, but haven’t seen in some time. I also sat down with a parent who became a friend, but someone I haven’t seen in a few years.
Always good to see friends, and worth making the time.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
It’s 2022. I would hope all software developers would learn that hard coding specific values in your system is more likely to cause issues than not. Across the years, we’ve learned not everything is installed on the c: drive, or that not everyone wants to put all data in a Documents folder (or in OneDrive). We’ve learned that any sort of magic number is poor practice, and we ought to know that hard-coded names are problematic as well.
Yet, we still see it happening.
This week I was reading about an admin issue in the Microsoft TechCommunity. This is related to Azure Managed Instance, but it’s really an In-Memory OLTP issue. That was introduced in SQL Server 2014, so I know the code for this was likely written in the 2011-2013 timeframe, but how can this type of issue get through code review and be released?
In this case, the name of a filegroup is set specifically to XTP. It’s a logical name, and I’m sure that some developer thought that things might be faster with a known location. That doesn’t make sense, and while this might not be an issue for most customers, I’m sure there have been some databases built with a filegroup called XTP. After all, there are companies named XTP. What about if this feature evolves to allow a second filegroup, maybe because of some distributed architecture need in the future? Are there then code paths looking for XTP or XTP2?
As much as possible, avoid coding values in your code that a user might enter as data. Names, paths, etc. Just don’t do it. Use variables, which are in every language, and let those values be read from the environment. This ensures that you don’t end up with weird support requests from customers because they chose the same value you did.
Steve Jones
The senior advantage means more to me all the time, though often I feel there are more disadvantages to being around longer than advantages. Getting older is hard, especially physically, and I struggle with that. Having more wisdom, more tolerance, and more patience, are good things, but I’m not sure I would consciously make that trade.
In a business, senior often equates to more experience and time working on systems. Sometimes it means more skill as well, but usually, the internal knowledge of how our systems work inside of our environment is more valuable, even though it can be hard to get management to realize that fact.
The last two years have had many technology workers at home, doing the same work they did in an office. Over the last year, there has been contention between some managers that want people to return to the office and many workers that want to remain remote. While lots of people don’t have a choice, there are some who do, and they are often the senior, skilled people. Lots of them quit, which creates a challenge for managers to fill their slots.
One would think that management would work to retain and keep more employees, but in some cases, the response is to find ways to make it easier to hire new people. One positive thing that happened is the relaxation of the rule for tech workers to have a college degree. That’s good for many people that are talented and don’t want to spend tens (or hundreds) of thousands of dollars on a degree. A little disheartening to me and others who think that this further devalues the experience of current employees.
There are some great employers out there, as well as some great managers that care about their staff. My company is one of them, and I appreciate how we run the business effectively while treating employees fairly. However, there is no shortage of poor employers who do not care much about their people, their training (or re-skilling), or whether staff leaves. They will continue to lean on whoever is still employed while replacing staff with cheaper, and less knowledgeable, new hires.
In thirty years, the one thing I’ve learned is that I need to be responsible for my own career. That means learning often, working on my tech skills, polishing soft skills, and burning no bridges. I need to be prepared and ensure I have opportunities and choices in the future.
That’s my senior advantage.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Recently we had the need to charge the car away from home, with some interesting experiences. I also had a question from my Mom on the Tesla.
This is part of a series that covers my experience with a Tesla Model Y.
A Long DayWe had a ski weekend planned in the mountains recently. I went up early one Saturday with one kid, who drove his fun WRX. His first time heading into the mountains, and I think the first time he’s driven us to ski in his car.
My wife had a horse class that day, so she drove the Model Y up north, going about 60 miles each way. I’d set the car to charge to 95% the night before, but her round trip ate up some battery, and as a result, she stopped partway on the journey to charge. She added 18kWh.
She and my oldest grabbed some food, walked dogs, and didn’t find it a problem to charge, spending about 13 minutes connected.
Winter StrugglesThe next day, she ran some errands in the mountain town and noticed the car was down to about 30%. She stopped at the Supercharger in Silverthorne, finding only one stall open. When she plugged in the car, it recognized the charger, but didn’t charge.
She tried a few things, but gave up. She was slightly annoyed, and didn’t want to wait to try another charger when someone left.
Between her driving and the cold, we lost about 8% of charge in subzero temps. Some of this was thermal protection, and some was likely additional inefficiency of the battery in very cold weather.
Getting HomeThe next morning, we woke up with 17% on the battery. I started the heaters to warm the car as we packed, as well as get the battery ready. We drove over to charge, stopping for coffee.
In 32minutes we added 42kW to the battery, which was more than we needed to get home, but it was what the planner recommended. Since we were chatting, handling some email, and drinking coffee, we weren’t that hurried. I might have stopped around 60% charge, but we let his get up to 72%.
The drive home was easy, and I had plenty of charge to run to the gym and a few errands later.
Charging TimeI chatted with my Mom about this a bit, as she’s curious. She has no interest in trading her Lexus for a Tesla, but she asks about how it works and how we deal with it. She hears a lot of FUD in the news about electric cars.
Here are our charging stats for 16 months.
My life supports charging at home, and the driving profile for me is that we almost never need 300 miles of range. Even a ski trip is about 220miles for me, so the car has plenty of range. I don’t tend to worry about it, and even on the ski trips, I don’t know if I can go up and back on a cold day with a full charge. I rarely charge about 95% and on ski days, I stop for 15 minutes to use the restroom, get coffee, and let the car charge.
In general, I just don’t think about charging the car. I have on trips, but I’ve also had to think about fuel on trips. We drive through Wyoming and Montana at times, and we usually fill petrol cars up when they’re 1/2 empty. We do this because the fuel is not as reliable in remote places.
The mountains of Colorado aren’t remote, but long trips require a little planning, and I don’t find that onerous.
I also just don’t think about range because I have a (mostly) full tank every day. With my gas/diesel cars, there are plenty of times we’ve run down the tank and have to plan a gas stop that day. I almost never do that with the Model Y.
I probably wouldn’t have written this post if not for the conversation with my Mom. While the charger not working was mildly a hassle, it was a 3 minute conversation with my wife, not something that was concerning.
For most people, electric works find. However, it is a different paradigm, and you should think about your lifestyle and driving profile. It’s not for everyone, but it is for me, and I really enjoy the car.
Today’s coping tip is to ask other people about things they’ve enjoyed recently
I asked the question on Twitter and on Facebook, looking for interesting responses from friends. The more fun ones:
- Got a dog (love this!)
- tabletop gaming
- cooking a meal with a kid (love the kids around the kitchen)
- woodworking
- bought running shoes (I can relate)
For me, I think a couple quiet dinners out with my wife were very enjoyable. I cook a lot at home, and we often have our kids with us. Date nights have been rare lately, so these were nice.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to challenge negative thoughts and look for the upside.
I’m struggling with some negative thoughts outside of work. This year as I coach older girls, they have many other interests, they’re distracted, less committed, less motivated. Not all, but enough that it’s tough to build cohesion and makes the job of a coach harder.
The upside is that it’s an opportunity for me to learn how I might approach things differently. I’m not sure what to do here, but I recognize there’s an upside, even as I’m unsure and frustrated, even a little down.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
The grade for January is a D. Details below, but just not making a lot of progress in these areas.
I set goals at the beginning of the year, and I’m tracking my progress in these updates during 2022.
Reviewing GoalsThe various sections are listed below, and I’m giving a SMART grade based on what I listed. Have I gotten much done. January wasn’t too busy for me at work, but a little busy outside of work with coaching ramping up and my wife being down from work for a few weeks.
I got a bit done, but not a lot.
ReadingTwo books on the list: marketing and coaching. I haven’t picked a marketing book, but I picked Wolfpack for coaching. With one started and one not, this is an F.
CareerMy goals:
- Set up a tracking system based on the book for my efforts with customers Track and calculate my scores to help me better approach how I work with customers. Review this with my boss and with someone in sales
Rating: D, I rated myself, but no review with others. On me for not getting this scheduled in a timely manner.
Community 2 speaking engagements for community (RMOUG Training Days and SQL Bits are on my schedule) Reach out to 10 SQL Sat groups that did not run an event in 2022 and motivate them for 2023. This should be at least 2 emails to each person/group. Reach out to all 2022 organizers and ask about plans for 2023 Start coding a tool for converting schedules to HTML. Hold office hours once in Q1 Send 3 monthly SQL Saturday updates to the community
This is a lot. It’s a lot for Q1, especially the emails, as I don’t want to overload any one individual. Two can be challenging. I set up a repo, I did reach out to 9 organizers from 2022, so that’s half. We also have 4 new events on the schedule and 6 previous ones. Given this is only 1/3 of the quarter, I think I’m on a C pace.
PersonalMy goals:
- Use my Power BI report for stats Update the Power BI report based on feedback from athletes Build 6 wooden coasters – I haven’t made enough time for my hobby here, so I’m going to start small. We need more coasters (as noticed over the holidays), so I want to build 6. Same style, different.
The first two are done, but it’s been in the 20s and 30s in Denver, so not a lot of enthusiasm for woodwork. Maybe a C here as I have updated the report and gotten feedback. No coasters though.
Does Context Info work across databases? This post shows it does.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. Here are some hints to get started.
The DemoSomeone asked the question, would a trigger in another database see context info from a different database. I thought it should work, but decided to test it.
Here I’m going to create a table and trigger in database compare2. This is looking for a context value.
USE compare2
GO
CREATE TABLE TriggerTest (myid INT, mychar CHAR(1))
GO
CREATE TRIGGER tri\_triggertest ON dbo.TriggerTest FOR INSERT
AS
BEGIN
IF CONTEXT\_INFO() = 0x1256698456
PRINT 'caught'
ELSE
UPDATE dbo.TriggerTest
SET mychar = 'X'
FROM inserted i
WHERE i.myid = dbo.TriggerTest.myid
END
GO
Now, back in DB 1, I’m going to set CONTEXT_INFO and insert a value into the database. This should give me a result where the trigger updates the table. A “normal” action.
USE compare1
GO
SET CONTEXT\_INFO 0x000
GO
INSERT compare2.dbo.TriggerTest (myid, mychar) VALUES (1, NULL)
GO
This does, as the table contains a 1 and X.
Now, same connection, let’s set the magic value for context and insert a row. Now the trigger should avoid the update, letting my bypass the normal action. This is what someone was trying to do.
SET CONTEXT\_INFO 0x1256698456;
GO
SELECT CONTEXT\_INFO();
GO
INSERT compare2.dbo.TriggerTest (myid, mychar) VALUES (2, NULL)
GO
When I look at the results, I have the “caught” message. The final results from the table are shown here:
As you can see here, the context is with the connection, not the database. The database doesn’t matter for this value, it’s whether or not the connection that sets the context (the session really) is still alive when it accesses the other database.
SQL New BloggerThis was a quick test for me to answer a question and prove this to someone (and myself). I thought this would work, but I spent 5 minutes devising a test. It took me less than 10 minutes to put this post together.
This shows volunteerism (helping someone), testing ability, and diligence to prove something I suspected was true. I didn’t assume, I tested. Lots of employers love that.
You can raise your brand and be a SQL New Blogger like this, showing your knowledge.
I had never heard of a vector database. I assumed this was a specialist type of database used for a particular problem domain, like a streaming database or graph database. There is a need for specialized platforms in certain situations, but I wasn’t sure what a vector was. The description I saw for a vector database was that they “… are specifically designed to work with the unique characteristics of vector embeddings. They index data in a way that makes it easy to search and retrieve objects according to their numerical values.”
That sounds like any database. However, I saw a few more articles on the hype and then some details about the ways in which this type of database is helpful. Essentially, this is a database designed to store the outputs from various Artificial Intelligence (AI) and Machine Learning (ML) models that examine unstructured data. Things like images, video, audio, and even text are turned into numerical values, or vectors. The vector database is designed to help index and then search these vectors.
What is interesting about the possibilities here is that the entire image, video, or whatever isn’t turned into a single numerical hash of some sort. Instead, the AI/ML process might identify that Steve Jones is in this video. That he is wearing a hat, or that he’s wearing a kilt. If I wanted to search for other videos of Steve Jones, or if this is the type of hat he’s wearing, a vector database can help. It’s much more powerful than simple tags that might be placed on a video because the details of the content are rendered into vectors which can be compared to other vectors. Not for exact matches, but likely ones.
One interesting example in the second link above is that content could be “vectorized” to determine if an apple in the content refers to a fruit or the company that Steve Jobs and Steve Wozniak made famous. Not easy to do with a tag, but more possible with a vector database.
And lots of data. Lots of vectors specifically, whose inventory is growing all the time. As more software is built to analyze unstructured data, and as organizations collect more unstructured data, the need to apply database techniques to this data becomes important.
For those of us working with databases, I’d expect a lot of the mechanics of dealing with a database would still apply. Things like security, backups, and indexing will be needed with vector databases. We’ll get calls about slow performance, missing data, or strange results, and we’ll troubleshoot the system. How we do that specifically might vary, but those are just details we’ll work out.
I like the idea of new databases, which provide more tools, challenges, and opportunities for us as data professionals. I haven’t met anyone using a vector database yet, but I’m looking forward to the day when that happens.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to decide to lift people up rather than put them down.
This is something I am trying to practice more as a coach, pointing out positive things rather than negative ones. Not that I won’t criticize, but the goal is to find places for improvement rather than point out issues as negatives.
People are hard enough on themselves with mistakes. I should note the action provides a way to move forward.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to take a small step towards and important goal.
One of my goals this year was to work on a way to score myself with customer interactions. I spent time at the end of last week setting up my scorecard in a spreadsheet to track things. A good thing as I have a customer call this morning where I’ll need to rate myself again.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
In a previous post, I set up the basic databases for the PoC project I’m working on. In this next post, we’ll get the Flyway Desktop projects set up for the PoC.
This is part of a series of working through Flyway and Flyway desktop to demo database changes. Disclosure: I work for Redgate Software.
Flyway DesktopFlyway Desktop (FWD) is the GUI that Redgate built on top of Flyway for managing your database project. This replaces the SQL Source Control and SQL Change Automation products that we used to try and integrate into IDEs.
I like Flyway Desktop, which is standalone app for capturing code and committing it to Git. It is a project based app, so you set up a project in a folder for a particular database (and possibly schema). In my case, I’m going to set up two projects to start for my PoC.
Version ControlWe work with Git, which has become the de facto VCS for most people.
On GitHub, I set up a public project where I’m putting this repository. It is located at: https://github.com/way0utwest/FWPoC
This is on my local machine as a copy at e:\Documents\Git\FWPoC
MSSQLThe first project I’m setting up is my SQL Server project, in a folder called MSSQL under the root of the git repo. This is a project folder just for Flyway Desktop. Once I create the folder, I’m going to create a new project in FWD.
I get a form after clicking New project. I’ll set the name as MSSQL, choose the root folder, and since I created the folder, I uncheck the checkbox. While I appreciate it’s good to put things in a subfolder, at times I’ve had FWD make a subfolder under the folder I choose, so I’m wary of this box. Mostly because I make mistakes.
Once this is created, I start in the Schema Model tab. I hate this nomenclature, as it’s weird. This is the list of objects whose code I’m capturing. There’s nothing here, because I need to link this to my development database.
If I click the “Link” button at the bottom, I get a dialog for the JDBC connection string. Don’t worry, you don’t need to know Java. Just fill in the boxes.
Two things in the dialog above.
One: Click Trust Server Certificate. Most new installations of SQL and driver upgrades require this. I don’t know why we don’t have this checked by default.
Two: Click “test connection” in the lower left, so if you have issues, this gets found quickly.
Once this is done, I go back to the Schema model and I see this:
I’ll select this object and save this to the project. This puts the file in the file system for this object, but doesn’t commit this to version control. We can see this in Visual Studio Code below. This is just a git repo, so if I open it in VSCode, I can see the file, the contents, and a note there are changes to be committed in the left icon.
One project complete.
PostgreSQLMy second project is for PostgreSQL. Same git repo, similar process. I’ll create a new project, but say this is a PostgreSQL project.
The process is the same. I’ll link this to a dev database. I need to specify the port and database I’m using. I also specify the schema here, as it’s not the default.
Once I get the project connected, I see the same as I did above for SQL Server. I save it, and I get a slightly different structure in the project. I see a schema below the schema-model folder. In here, I see my table, but it’s not the code, but a description.
From here, I just commit and push this stuff up to the repo. Note that commits and pushes, can push everything from both projects as they are in one repo. I did this on purpose to keep everything organized for me. However, if this were a team, I’d likely separate SQL Server and PostgreSQL into separate repos so individual developers don’t get confused.
The next step here is to get a second database to where I can deploy changes for each project. I’ve got these set up, and in the next post, we’ll work on an initial deployment.
Monitoring your SQL Server instances is important to ensure you can meet your SLAs. Availability, performance, reliability, quality, whatever you care about, it’s important that whoever is responsible is looking at how the database is performing. At Redgate, we have multiple teams working on SQL Monitor to enhance and grow it to meet your needs.
A short while ago there was an internal conversation recently about page life expectancy. We’ve had some customers ask about this and setting alerts to watch this value. Our developers and sales engineers asked for a few thoughts from Grant and others on how to respond. There are a variety of opinions, some saying monitor it, some saying don’t bother.
I think both pieces of advice have merit, which is to say that this isn’t a metric that you can look at in isolation. There is no value of PLE that is good or bad, or that says x is wrong or y is right. There is both a subtlety and a complexity to understanding what PLE is telling you about your system. If PLE is growing, you have to look deeper. If it’s falling, same thing. If it suddenly drops, there are multiple possible causes, and you need to examine other things. However, in many cases, this isn’t an actionable metric, but one that provides context about what might be happening in the database when combined with other values you monitor.
This certainly isn’t a metric that you want to set an alert on because it can rise or fall and many times the change isn’t indicative of an acute problem.
This is just one metric of many that are available in SQL Server, and knowing which ones to monitor is something good administrators learn. They know that very few values they instrument have a good or bad value, and often the rate of change needs to be combined with the actual reading to determine if there is a problem. We also often want to know if a high (or low) reading appears for an extended period of time. Having 100% CPU being used for 3 minutes likely isn’t an issue. If it lasts for 3 hours, I might feel differently.
Metrics have more complexity than just having a range in which we ignore them and a limit at which we alert people. They are intended to be combined with each other, with observations by clients, and with the experience of looking at past observations over time. Our systems often develop patterns, and we don’t get too concerned about any values when the pattern repeats. It’s when something new happens and someone complains that we dig in to determine if there is a problem or the start of a new pattern.
We definitely need monitoring of our database metrics, but we also need to understand why values move and the implications of them doing so. That’s something which isn’t as simple as setting alert for each one based on some value we think should never be exceeded.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to plan something fun and invite others to join you.
The fun thing is actually skiing today. My wife and I wanted to get some time away with our kids, so we invited them to come to Keystone for Sat/Sun/Mon for a getaway. They’re joining us, though some are leaving tonight to return home for work tomorrow, while we’re staying another day.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
The #SQLFamily is amazing, at least I think it is. Like many families, it’s welcoming, supportive, and comforting. It’s also maddening, frustrating, and exacerbating at times. Like most families, or at least the ones I know, it’s not perfect, but it’s what we have and at the end of the day, most of us get along with each other.
It’s also an open group of people. In general, we welcome people with open arms and smiles. Those of us that are more visible or prominent are willing to listen to, help, and support anyone. I was overjoyed during the recent PASS Data Community Summit, where I had the chance to see so many people that I haven’t seen in person in 2-3 years. I met many other interesting people for the first time and enjoyed the experience.
Not everyone feels the same way. I loved seeing Kimberly Tripp and Paul Randal for the first time in years and was honored to share the stage with them for the Community Keynote. I enjoyed the time we spent together, but afterwards Paul wrote about some people not feeling included or welcome.
I understand that feeling. In many ways, much of adult life can mimic teenage years in high school with cliques and pettiness. I won’t pretend that doesn’t happen in the #SQLFamily, but I find it quite minimized, especially compared with many other communities of which I’ve been a part in the past. I haven’t seen the higher profile speakers and leaders in the SQL Community dismiss someone for asking a question or expressing an opinion. I have, however, seen that in other communities.
It can be intimidating to walk up to someone that you don’t know who you might feel is famous or well known. It can be intimidating to just walk up to a group of people who are talking when they appear to know each other. I have that feeling at times even today, so I appreciate feeling like an outsider. In the Summit keynote, I talked about the thrill in meeting Kalen Delaney in 1999 and shaking her hand. I was nervous and intimidated to ask her a question after her presentation. At the time I hadn’t delivered a talk in front of anyone outside of school environments and was a fairly introverted geek. It was hard to step up and make that effort, but I’m glad I did.
As Paul writes, anyone is welcome in the #SQLFamily. Anyone can join. You don’t have to come shake a hand or say hi, but I’m happy when you do. Will we be best friends right away? Probably not. Will we go out to dinner that night? Maybe. I’ve certainly met attendees at events and then had dinner with them. I know plenty of other speakers who have as well.
Many of the people who speak, organize, and write/blog/tweet/etc. in our industry are friends. We do value time with each other, and that can feel like a club, but it’s not. We enjoy seeing each other and want to catch up, like any group of friends. However, we are also welcoming of newcomers, so feel free to introduce yourself.
Ultimately those of us who engage in these highly visible, extroverted acts are often just like the rest of you. We’re a mix of people that are mostly introverted, with a few extroverts thrown in. Some speakers are very smart and talented, some are more like me: we know enough to get the things done that we’re asked to do. Some of us love to go out and sing karaoke until all hours of the night and others prefer a small dinner or a little tabletop gaming in a quiet environment.
My encouragement to get people to meet others, network, set up events or meetups, and more isn’t to try and convince any of you to join the cool kids club. It’s not to get you to change who you are.
It’s to help you find your tribe. To find your kind of people.
I’d love to greet all of you with a hug at events, call you by name, and go out to dinner with you. I can’t because there isn’t enough time in the day. And quite frankly, I really, really value my alone time. What I want more than anything is for you to be successful, find a great job or career you relish, and for you to enjoy spending time with those you enjoy. Whoever and wherever that is. That takes some effort, but it’s worth the energy involved.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to try something new to get out of your comfort zone.
I did two things here. First, I’ve been participating in a Jan American Cancer Society Challenge and fundraiser. I’m doing this because cancer sucks, and I wanted to help. I rowed in college, but I don’t want to just focus on rowing, so I’ve been trying to fit this into my normal routine.
However, I decided to go longer and make this my exercise for a few days. Part of that is being busy and bad weather, part to change things up. I pushed harder and instead of a 10-12min row, I went 22 and 5k.
Second, I asked my son to cook a recipe. He showed me something he found interesting, which usually results in me asking for the URL and I shop and cook. This time I asked him to make it, which is slightly uncomfortable for me. However, I’ve been extra busy, so I got out of my comfort zone.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to focus on what’s good, even if today feels tough.
Yesterday was a tough day. I got busy, distracted, had some bad news come up, family needed some care, and I struggled to get work done. On top of that, something I thought would be easy to setup and test for a customer didn’t work at all. I’d made an assumption and my process needs to be redesigned.
I felt like I wasted a day.
However, it wasn’t all bad. I took care of my wife, who had surgery recently. I got her lunch and spent a little time chatting with her. I helped my daughter get a few things done before leaving to return to school. I got the dogs outside for a walk and I found time to exercise.
There were good things, even if it was a tough day.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I saw this article a few months ago, which talks about engineers at Facebook not knowing where their customers’ personal data is stored. The engineers were being questioned in a legal matter, where they were asked to definitively state where all personal PII data for any human was stored by Facebook. Their answer was that they didn’t think anyone in the company would be able to answer that question.
Facebook has been controversial over the years, and plenty of people dislike the way the company conducts business. I noticed no shortage of data people (and many others) commenting on this situation, saying that Facebook should be shut down because they don’t know where data is being stored.
However, I don’t agree. In working with lots of customers, on all aspects of how they handle, process, and manage data, I expect this to be a problem in many organizations. Whether large or small, whether they have few or many software engineers, it is highly possible that there isn’t a good list of where personal data is being stored. As we work with customers to classify data with SQL Data Catalog, that process takes a long time, and very often the system administrators or developers who undertake take the task are unaware of all the places where data is stored.
That’s just in relational databases, ignoring all the Excel spreadsheets, text exports, mail merge operations, and uploads to services for mailing, analysis, or something else. Very often the control of personal data is fragmented among groups, with there being few efforts made to coherently manage a customer’s data.
The world has adopted computing at an incredibly fast pace, often by people with little knowledge or forethought of the implications of gathering and processing data. In many cases, probably most cases, there is no overriding strategy. Just like with applications slapped together quickly, we find data being gathered and stored based on the requirements and demands of business people, with no planning for management or archival, and often not even with any security requirements.
I liked the GDPR as a step forward, asking companies to not only handle data appropriately, but remove it when not needed, not use it without consent, and to be able to keep track and delete it if not necessary. I don’t know that this has been successful, but it has changed handling practices in some organizations. At least in responsible organizations, and many of them have had to track down personal data to delete it. I’m not sure they know where it all is, but I at least assume they know where all of the data about a person is in their various relational stores.
As a technical person, do you know where all data is stored about a customer? Are you sure you know where marketing has been keeping information and what other mailing, analysis, reporting, CRM, etc. systems they’ve put data? Any idea how many copies the operations group keeps? Test systems, QA, UAT, and others? What about test data sets, are they sanitized? Perhaps legal or finance has gotten extracts of data to reconcile their systems.
Tracking down all data can be hard, and I’m not surprised Facebook struggles. I would guess engineers in many organizations would have similar answers.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
SQL Data Compare (SDC) is a great way to sync data among tables. It’s a software utility analogous to SQL Compare, but working with data rather than schema. I had a customer ask recently about setting up a SDC project and then calling that from the command line rather than using the GUI and clicking.
This post looks at how you can call a project from the command line. The project has a WHERE clause in it, so it uses the settings from the project.
We have the data shown here, from two different databases. There is 1 row in the first table that is not in the second table (in the second database).
I’ll build a SQL Data Compare project. In this project, I point to these two databases and the tables.
If I edit the project, I can choose the tables and views tab. Here I see my tables, and I select the dbo.RSSFeeds table.
When I select the row with dbo.RSSFeeds, I can then click the “Where clause” option and get a dialog where I can filter data. Here I can enter the where clause I used in the first query above. I also have the”use the same WHERE Clause” box checked.
Now I can save that project. I’ll then execute this from the command line. Note that I don’t have the SQL Data Compare install in my path, so I qualify both of these files, the executable and the project file. The call for me is:
"C:\Program Files (x86)\Red Gate\SQL Data Compare 14"\sqldatacompare /project:"C:\Users\way0u\OneDrive\Documents\SQL Data Compare\SharedProjects"\DLM\_Demo\_RSS.sdc
You can see this being run below:
I can see there is a single row in the DB! that needs to move to DB2, which is the result I saw in the first queries above and in the SQL Data Compare gui.
If I add the /synchronize option to this call, SQL Data Compare will deploy the changes. Once I do that, I can query the two tables and see the data is the same. At least the data matching the WHERE clause.
Some of this is documented, but not worked through in an example, so I wrote this post to help myself and anyone else looking to work with SQL Data Compare from the command line. This is a great way to sync data easily between systems, if you have a repeatable set of data that you need to move.
SQL Data Compare is a very handy tool for checking and moving data between tables that needs to be synched. All sorts of lookup or reference data can be managed with SQL Data Compare. If you haven’t tried it, grab an evaluation and give it a try.
Disclosure: I work as an advocate for Redgate Software.
Today’s coping tip is to be gentle with yourself when you make mistakes.
I forgot about a commitment. I had agreed to do a webinar and prepare some content. I got busy with life, almost forgot the webinar, and was scrambling to put together a couple of slides. I got something ready, but it didn’t look great, and I wasn’t happy with it.
Fortunately, I never needed the slides in the webinar.
I was upset with myself, and I spent a few minutes berating myself, as well as thinking “what could I do differently?” Should I set more reminders? Do I need a better to-do list? Should I let some personal things go to ensure I stay on task?
Ultimately I stopped that. I had planned for the webinar, it was on my calendar, and I had done some prep the week before. I got hung up this week with a few personal issues, and that is something I need to accept will happen. Things worked out, so I need to forgive this mistake.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to get outside and notice five beautiful things.
I decided to do this on a snowy, stormy day in Denver. I was up early with my daughter running a couple errands, and I took a few minutes at different times to look around. Here’s what I noticed:
- My driveway markers stand out nicely and helped us get out of the driveway, which was covered under a blanket of snow.
- The snow is sticking nicely on the trees, making a wonderful winter wonderland.
- It was a bit of a blizzard, but walking between the car and a building, we were sheltered. Without wind, even in 25F weather, the air was brisk and refreshing.
- Snow is fun. Even without sticking together to make a snowball to throw at my kid, it’s a neat form of water.
- It was mostly cloudy, which made it hard to see, but a few times the sun started to shine through. Even mostly blocked by clouds, the bright ball made the entire landscape brighten up.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I’ve had a goal to redo my demo environments and get them set up to work for a variety of customers in different places. I decided to do this in a way that uses new Redgate technology, with the integration of Flyway with Flyway Desktop.
This first article looks at the environment I’ve set up for my system.
This is part of a series of working through Flyway and Flyway desktop to demo database changes.
OverviewI wanted to demonstrate DevOps, as I would recommend most customers set up to get started in their environment. It doesn’t matter if they’re SQL Server or another RDBMS, the approach at a high level is the same. Obviously the setup for each technology would be different in the details.
I DO NOT recommend starting with a live database or project. This is a Proof of Concept (PoC), so use something that can fail.
For this start, I’m working with SQL Server and PostgreSQL.
SQL ServerI work with SQL Server all the time. That’s the majority of my customers, so we’ll start with a new SQL Server instance. I’ll run this code, but this is to simulate multiple environments:
CREATE DATABASE FWPoc\_1\_DevCREATE DATABASE FWPoc\_2\_IntegrationCREATE DATABASE FWPoc\_3\_QACREATE DATABASE FWPoc\_4\_StagingCREATE DATABASE FWPoc\_5\_ProdGO
These environments are set up to be this model:
- 1_Dev – the place I make code changes. This should be the only place I actually touch code.
- 2_Integration – this is a place where we mix up code from multiple developers. all code ought to get pulled to each dev db at some point, but for many environments. I recommend getting some deployment here for devs to see all code.
- 3_QA – standard test environment
- 4_Staging – This is a DBA test environment, and this ought to get refreshed from production, either schema-only or full refresh, to validate the deployment
- 5_Prod – live database.
I’ll then run this code:
USE FWPoc\_1\_DevGOCREATE TABLE dbo.Demo (DemoID INT )GO
The idea is we get that table to the other 4 DBs without actually connecting to them directly and running this code.
PostgreSQLMy setup for PostgreSQL will be similar, but smaller. I’m experimenting here, and one large 5 environment demo is enough. Here I’ll use 3:
- fwpoc_1_dev – development environment
- fwpoc_3_qa – test environment
- fwpoc_5_prod – live environment
Again, the goal is only write code in 1 and get it to 3 and 5. I’m keeping the numbering to try and keep everything simple and similar.
I want to run PostgreSQL, but I want to use containers. I have enough server services running, so I’m starting with a container. First step, update the container:
Next, I need to run the container. I’ll do that with this command. This names my container pgdev and gives me a password to connect for the “postgres” user. I also will use a volume on my local drive.
docker run --name pgdev -e POSTGRES_PASSWORD=demo1234!@# -d -p 54320:5432 -v C:\Docker\postgresql-1-dev:/var/lib/postgresql/data postgres
Before I run this, I’m create folders on my local C: drive for the docker data to safe. This is the folder I’ll map in my containers.
I’ll connect with Azure Data Studio (ADS) as I have the PostgreSQL extension. Once connected, I’ll query the information schema tables.
This works. Now, let’s set up a dev environment similar to SQL Server. First, we create a database with the CREATE DATABASE command.
create database fwpoc_1_dev
Once I run that, I’ll select it in the ADS connection drop down. Now I run this to create a schema and table.
create schema poc;
create table poc.Demo ( DemoID int);
insert into poc.Demo (DemoID) values (1), (2)
select * from poc.demo
This works and gets me a development environment. I’ll start another container for qa and prod, but I won’t do that now. Instead, I’m just getting the base environments set up.
SummaryThat’s it. This post was about getting an environment set up and ready for development on a PoC. This is the first step, and it’s already a lot.
Future posts will look at the Flyway and Flyway Desktop settings, a repository, and a MySQL set of environments.
Follow the entire series on my blog.
We collect a lot of data in our databases. Not as much in bytes as a lot of the video/audio/TikTok/Instagram sites, but still enough that many of us are constantly adding storage to our systems. All this data is not only a challenge to manage, but it also means that we are regularly dealing with query tuning issues. Better code, indexes, and more become regular challenges with large volumes of data.
I am a big fan of trying to reduce the data you manage where possible. Archive, delete, remove older data, do something. This not only makes your systems easier to manage and improves performance, but it reduces your risk. Any PII data you have that might store is an ongoing risk in the event of a data breach. I don’t pretend this is easy to do in any way, but it’s a good idea.
If you can remove data (or must because of a regulation like the GDPR), how do you ensure that data is deleted? Most of us know how to submit a DELETE statement, but that just removes the data from an online system. What if you restored or recovered this database tomorrow, would you remember to delete the data again? What about losing a copy of the data or log backup? What about older dev/test systems that were refreshed from production? The data might be in there. If you work through the possible problems, deleting data from a system isn’t as simple as you might expect.
This might be even more complex in the age of cloud computing, where we don’t control the hardware for primary systems, or for backups. There is an article on deleting data in the cloud that talks about the government standards that require that you not only delete data, but that you overwrite the physical hardware to ensure it can’t be recovered. This still doesn’t address backup systems, but it does help to clarify that many of us might start to demand cloud vendors not only de-allocate the disks we use (or the backup storage), but they also overwrite the storage with zeros.
Data security and the risks of not taking this seriously is becoming a bigger issue all the time. I don’t know that poor security will cause your organization to fail, but there can be significant costs and possibly reduced employment opportunities. While you might not want to be overly paranoid or concerned about every possible issue, it is worth asking questions of vendors, working through likely scenarios, and trying to quantify risk.
More and more systems are regularly under attack from malicious groups, which means we want to minimize simple mistakes, reduce human error, and limit the exposure we have from the data we have by storing only the data we need.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to eat healthy today with some nourishing food.
A few days before think I was getting ready for my daughter to leave for university. I enjoy cooking for everyone, but I wanted to get her something special for dinner. I decided to take this ramen recipe and then add a few new twists.
I added:
- baby corn
- broccoli
- chili garlic sauce on top
I removed:
- sugar snap peas
- avacado (no ripe ones).
It was a good, nice, healthy meal for everyone.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
It couldn’t have happened to a worse set of people, and I’m glad it did, but I’ll still take a positive lesson out of this. A hacker sent a typo in a command to a botnet and lost control. That’s kind of funny, and I’m glad it happened. The less botnets, the better, IMHO.
The coding in this software didn’t have good error handling, which is a lesson in and of itself. Overall it seems many developers do a good job of error handling, but I still encounter more pieces of software that allow problematic input than I’d like. While we don’t have great error handling in T-SQL, you can make some checks, and you should.
That’s not the big lesson for me. The bigger lesson is that we ought to do less typing in much of our daily work. The last decade has had me work often with companies looking to implement DevOps software pipelines and driving automation wherever possible. We want to limit the chances humans can make mistakes, which means we want to limit their typing. Or clicking, as is the case in much of today’s software.
Instead, we want to ensure all our code or commands are reviewed by someone, they are submitted to an automated pipeline, and they are validated or practiced on some system ahead of production execution. We ought to do this for no other reason than we want to ensure we have an audit trail, but preventing typos is good as well.
I don’t know if you can completely get away from typing, but we can reduce the number of human error mistakes if we include some static code analysis (including for commands), some peer review, some sort of unit testing, and pre-production deployment. A lot of mistakes I find are fairly simple ones. Common human error that occurs because we’re busy, we’re stressed, we’re moving too fast, or we just miss something.
Use the computer for one of its strengths. Tediously checking the simple things that humans do wrong.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to take a different route today and see what you notice.
I’ve had this tip come up a few times and each time I’ve enjoyed looking at the world from a different perspective. I tend to follow the same 4-5 routes over and over, often playing the “how-little-energy-can-I-use-in-the-Tesla” game.
Today I took a new route to coaching practice. I’ve gone this way a few times, but almost never, so I decided to follow along and see what I’d see. In this case, I’m leaving Parker and heading near Centennial Airport in Colorado, but rather than my normal Parker Road to Bronco’s Parkway, I turned off early, went between a few neighborhoods and then threaded through a bunch of new commercial/industrial construction around there.
What I was amazed with was the amount of new construction. I’d seen some new buildings from my normal route, but driving through there it looks like a tremendous amount of new buildings, especially warehouses, being built in a large area that used to be open fields.
Good and bad, but something I noticed for sure.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to switch off tech an hour before bedtime.
I read at night often, so this is a challenging one. However, I decided to give this a try, especially as my wife has been reading more lately with less TV.
To fill the time, I decided to put my phone on a charger and play guitar for an hour before turning in. My fingers aren’t quite as ready for this as I would like, so it ended up being a bit of playing, some rest and chatting with my wife, then repeat.
Not sure I love the idea of getting rid of all tech, since I read on a device, but removing most of the distractions from browsers, social media, etc. was nice.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I’m speaking at VS Live in March 2023 for the Las Vegas show. This time it’s at Planet Hollywood, which is a hotel a new place for me. I’m looking forward to seeing lots of friends at the show. I was in Redmond and Austin last year for the VS Live shows and I can’t wait to get back to Las Vegas.
If you want to join me at fun show, register with the code Jones for $500 off the price.
VS Live is a multi technology show. I always learn something interesting there from Brian Randall, Rocky Lhotka, Bob Ward, Andrew Brust or any of the other amazing speakers. There are lots of development sessions, both technical and soft skills, along with a few of us data people delivering talks on how to deal with the database and power your software.
The schedule is up and it looks exciting. A wide variety of talks, including these two from me:
- Architecting Zero Downtime Database Deployments
- Adding Graph Structures to your SQL Server Database
There are lots of talented speakers and great topics, so think about kicking off your 2023 development efforts by coming to VS Live Las Vegas, Mar 19-24, 2023.
Register with the code Jones for $500 off the price.
I’ve been doing a bit of work with PostgreSQL as part of my work with Redgate. PostgreSQL is a relational platform that is open source, free to use, available as a supported commercial product from various companies, and has been in active development for over 35 years.
More and more organizations are looking at PostgreSQL for relational data stores as both SQL Server and Oracle are very expensive, and this is a viable alternative.
This post looks at the basics of getting started with PostgreSQL by connecting to the platform with Azure Data Studio.
If you haven’t installed PostgreSQL, or want to work with a container, check out my previous post: Creating a PostgreSQL Docker Container with a Volume on Windows
Azure Data StudioAzure Data Studio (ADS) is a fork of Visual Studio Code, but specialized for databases. I don’t use it a lot, but I find it useful for some coding. They also have added an extension that allows connections to PostgreSQL.
If you click the Extensions blade and then search for postgresql, you’ll find it. You can see it installed on my system below. Before you do this, the button that says uninstall would say install.
Once installed, the connection is like any connection in ADS.
First, make sure you have PostgreSQL installed. If you want to do this in a container, see my link above.
ConnectingTo connect, first open a new Query window.
At the top there is a “Connect” button. Click this.
Now the connection blade opens. As you can see below, there are now two choices for the connection type: SQL Server and PostgreSQL. Choose PostgreSQL.
I choose localhost, as that’s where my instance is running. I also change the authentication type to Login and enter “postgres” as the default user. If you haven’t set up a user, use this. Then enter the password.
One last thing, as I often run databases in containers, I’ll change the port. To do that for PostgreSQL, click the “Advanced” button. I often use 54320, 54321, 54322 for my ports, so you can see I’ve entered that below:
Click Ok if you’re added a port, and you should see these details:
Click Connect, and you should connect immediately. Then you can verify your connection with a query or two. As you can see, the database is defaulted to postgres and my queries have returned the version and tables in this database.
That’s it. Now time to work on some pgsql.
A while back I ran across a blog post that talks about the difference between database snapshots and database backups. There are certainly some similarities and some differences, as well as an overlap in the places where you might choose to use each of these technologies. Both might be useful as a way to recover from a bad code deployment, but both aren’t necessarily helpful for a DR situation where the primary server has a catastrophic hardware failure.
It’s often the case that we learn only a bit about some of the technologies in SQL Server. That’s understandable as the platform has grown very complex, encompassing a vast array of technologies and options. There is often some overlap between them and possibly different places where you might choose to use one or the other to solve a problem.
In plenty of cases, one technology will stand out, especially when you have gathered enough requirements to understand the entire situation. More information can help you narrow down your choices, and even make a decision, if you know both (or all) the technologies well.
If.
The key here is that as you look for solutions, you should be sure that you understand the technologies well. Dive deep into each of your choices and make an effort to determine the positives and negatives, the advantages of one over the other, as well as the limitations or holes that may be present. You can work often work around limitations, but you should be aware of what they are.
We can’t learn everything today, or even everything that we need to know in a short period of time. However, when we are faced with a situation that has multiple solutions and unfamiliar technologies, we should ensure that we try to learn, ask questions, and do our best to understand the boundaries of the question and the technologies that might solve our problem.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to say positive things to the people you meet today.
In general, I don’t encounter a lot of people on any day, but I do get to see my family, usually someone at the gym, and maybe a cashier in a store.
Today I ran into family, and had the chance to smile and complement them. I went to a basketball game with two kids, one of whom bought the tickets. I made sure to thank him and tell them how I enjoyed the time.
While at the game, I thanked the vendors for their help, donated to their cause (they use tips for their server organization), and be happy and positive when making an order, really a request.
Managed to get to the gym as well, and ask the reception people how they were and wish them a good day.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to get moving. Ideally do something outside.
Taking time to walk the dogs outside, heading to the dog park to get them, and me, some exercise.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I’m honored to be heading back to SQL Bits 2023. I was selected to deliver one session, in the professional development track. With so many people submitting, I’m not surprised that this was the only selection.
This year, SQL Bits is in Wales, at the International Convention Centre just north of Cardiff. I’ve been to Cardiff only once, and am looking forward to going back and exploring a bit.
SQL Bits has been my favorite conference over the years. Others are great, but I have always loved this one. If you want to join me, start making some plans and register today.
Hopefully I’ll see you there.
In my experiments with the Flyway CLI (fwcli), I’m finding some interesting behavior, some of which is catching my by surprise.
This post looks at the baseline command and the issues with not having one. I also cover a naming issue. This is a bit long, but I wanted to document what happens as I experiment. I’ll condense down what a baseline does in another post (or two or three).
The ScenarioI created a FWTest database and put a small table in it. I can see in my Object Explorer (OE) that there is just one table. Assume there are no views, functions, etc.
My flyway.conf file points here, and if I run Flyway Info, I see the results below. The only important part is the “schema version”, which is empty, and the table, which shows one versioned migration called getone.sql.
The migration is in my SQL folder, which is below the location of the flyway.conf file. The configuration file is in the smoketests folder and it has only two lines uncommented: 1 for the SQL Server connection string and one for the location of the migrations, which is the SQL folder. The relevant lines are:
That’s the basic start. I’ve got a database, and I created a new script. The script in the file is shown here:
Running FlywayNow, I have a migration script I want to apply to a new database. What happens if I run flyway migrate. Will this create my procedure? Let’s see.
The output shown below runs and give me a green line and a red like. One success, one error.
The success is that the script was named correctly, so it passed validation.
The error is that there is no flyway_schema_history table. This is where all Flyway activity is tracked inside the database. Without this, there’s nowhere to stick the data on script execution.
The error does note that we need to run flyway baseline or set the baselineonmigrate option to true. The default for this is false.
Adding a BaselineLet’s do the baseline thing. The documentation for baseline is very poor (as of Dec 2022) in my opinion. I’ve sent a few notes around the company, as I think this needs to be cleaned up.
In any case, this will do a couple things. First, it creates the flyway_schema_history table (under dbo for SQL Server) as the place to store history for Flyway. Next, it will add a row as the baseline version for this database. I don’t have a baseline script, a “B” script, but that’s OK. I don’t need it for now.
Let’s try this. I’ll run this on my database and we see the results below. I’ve captured the text, ignoring the licensing and connection string part. The results are really here:
Creating Schema History table [FWTest].[dbo].[flyway\_schema\_history] with baseline ...
Successfully baselined schema with version: 1
This shows me we have created the table and added a baseline of version 1. If we look in the database, we see this: the new table, but no proc.
If I run flyway info, I see these results (again, ignoring the licensing and connection stuff).
What I see in here is that my database version 1 is baselined with no scripts. However, the getone.sql script is noted as a versioned script but ignored because of the baseline.
The baseline is supposed to be a level that includes all scripts up to that number. It’s not well explained in the docs, but this means that any scripts up to the baseline are assumed to have been executed in the database. This is the baseline. The baseline.sql script also is supposed to include the contents of all previous migration scripts, but as my test showed, I don’t need that.
The Problem with BaselinesThe thing I have to know here is that this baseline version means no scripts at this version or lower will be executed. Since my script was a v1 script, it gets ignored, as you see below:
Why this says 2 migrations, I don’t know. There’s only one script. I assume this is counting the non-existent baseline script.
However, nothing migrated, and my procedure isn’t created.
Be aware, that you want baseline scripts and other scripts to have discrete numbering.
SummaryGetting started with Flyway means we need a baseline to get going. We can do this without a script, but we do need to run flyway baseline, or set an option. I’ll look at those two items in a future post.
I also need to be careful with naming of scripts, as a script that matches the numbering of the baseline will not get executed.
The US was hit with a number of storms over the Christmas holiday weekend. This disrupted air travel for many airlines and their customers, but one of the worst hit was Southwest Airlines. They accounted for most of the cancellations, over half of their scheduled flights at one point.
A number of places reported talking with Southwest employees who blamed the lack of tech investment by Southwest over time, noting this caught up with them. The Chief Operating Officer disagreed, saying that their scheduling system is the best in the world, even as the CEO noted that their scheduling software couldn’t keep up and they fell back to manual operations.
Most of us likely have no idea of how Southwest software works or the scope of the problem. This airline does tend to operate differently than many others in that they mostly fly point to point, rather than using hubs. Possibly they have the best point-to-point scheduling software in the world, but it still couldn’t keep up with the storms covering much of the US.
There’s an interesting perspective on Facebook, supposedly from a pilot with 35 years of experience with SouthWest. If you don’t want to click, his view is the hands-on CEO retired years ago and accountants were appointed as CEO and COO. They improved the money flow, but neglected investments in tech and weren’t aware of how the business really runs day to day. The infrastructure and software deteriorated, and they’ve had many small issues, but issues that were bigger than other airlines. They’ve started turning around with a CEO that is more hands-on, but they’re digging out of a hole.
Like many of you, I’ve built and operated software over the years. I sometimes realize just how hard it can be to keep up with the demands of customers for adjusting how our systems work. I also know that it’s easy to slow your investment in a system that appears to works and limit your efforts to just maintenance work. Allan Hirt wrote about this.
This does bring up the issue of investing in systems and maintaining them over time. I see why many companies would prefer to purchase software and let someone else manage the investment in ongoing development. I also know that for companies that see software as strategic, likely there needs to be regular investment, upgrading and refactoring code, as well as finding ways to scale higher and use resources more efficiently. Especially for databases.
The battle between enhancing software and reducing technical debt is a constant one. I see this struggle being one that project managers and developers never agree on, but in the companies that seem to thrive, there is a balance. Perhaps it’s splitting the sprints, perhaps it’s allocating regular time during each development period, or maybe there’s another way.
One thing is certain. We need to find a balance. Otherwise we might get into the situation where a complete rewrite or replacement of software is warranted; a situation that is almost always very costly.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to learn something new and share it with others.
I do this all the time. It’s a part of my job, and I did this recently with a PostgreSQL container. I also did this with a piece of my coaching training, helping kids understand where to serve from /to for better chances of success.
Sharing is a great way to ensure to engage with the world and give back.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Many of you reading this probably work primarily in SQL. Even if you are a developer whose main language is something else, you write a lot of SQL. Even if you have an ORM writing the SQL that goes into production, I bet a lot of you are writing queries against a database to check that the data coming back in your application is correct.
As for me, I mostly work in SQL, with PowerShell and Python being second and third. I tried R for a while, but I think Python does everything R can do and it’s much cleaner. I find R very cumbersome. I rarely write C# or experiment with anything else, but that’s the nature of my job. PowerShell is important, as I do a bunch of DevOps and PoSh is a good choice to work with on the command line for gluing processes together.
There was a set of the top articles on programming languages from 2022 that I saw recently. I found it interesting to see what was popular. The top one was about Python being the most popular, but it shouldn’t be. This one feels like clickbait, and I find many of the conclusions not making an argument against python in a meaningful way.
There are some other links on the “hotness” of various languages. I think these are clicked on as many developers are just curious about what others are doing, and what they might experiment with. While I like curiosity and experimentation, I do think that many of our important systems in organizations need to be built with mainstream technologies. Support and staffing are a challenge, and while Golang might be great, finding people to read and code in it is hard. I don’t know how to balance the growth of new tech with the safety of old tech, but I wouldn’t stray too far from the mainstream for anything important.
I do find it interesting that COBOL makes the list. I know there are still lots of COBOL systems, and while there aren’t a ton of jobs, there are jobs and little competition. If I were 10-15 years younger, this would be tempting. Of course, I’d have to be willing to adapt to the jobs, but it is tempting. I know a few people making well into the six figures because of COBOL jobs.
It’s nice to see SQL is one of the top 10 languages in use, according to this survey.. It was #9 in 2021 and #8 in 2022. I don’t know it grew in popularity so much as assembly declined compared to other skills. I certainly can’t see SQL going away, but it’s not as popular, clickbait-y, or exciting as other languages. Instead, it’s a core, required skill for any serious software development. Whether you use relational or NoSQL databases, likely you need some SQL skills.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to take five minutes to sit still and breathe.
Sounds simple. I find this to be very challenging. If I’m just sitting here, why am I now getting something done? I could be productive during this five minutes.
I struggle to do this, but I’m taking this time to do this. It’s a few days earlier in the week, and I’m at a customer site for a bit of work. During the time someone else was working with the customer, I took five minutes to sit in the lobby of as building. I set a timer and put my phone in lap. I then sat, looked at the fountain for a bit, closed my eyes a bit, and tried to just sit.
It took a few minutes to relax. I kept thinking about things I needed to do, reflect on the customer visit, and more. After a bit, I could quiet my mind and just relax.
It was nice, though I still felt like I could have been productive during that five minutes. Clearly I need to practice this more.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to write a list of things you feel grateful for and why.
- My wife – our relationship continues to grow, and I cherish this after more than 27 years.
- Health – no major issues for me
- Finances – secure enough to not worry, with relatively minor stresses over expenses.
- Children – healthy and doing well in life. I’m proud of them
- Coaching – I love working with kids and am grateful for the chance to do so.
- Animals – The dogs, cats, and horses enrich my life. I’m grateful, even when I’m annoyed by something.
There is plenty more, but this is a short list.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
In a previous post, I got Flyway installed as a CLI utility (command line interface). This post will look at the first connection to a database.
If you need to install Flyway on Windows, see my previous post.
ConceptsFlyway is a command line executable that takes various parameters to control what it does. Essentially, these are command verbs and decide what action is running. The previous post looked at the version command.
There are also parameters that can be included before the command. There are lots of parameters.
However, much of the way Flyway works is controlled by the flyway.conf file, which is inside a conf folder where you installed Flyway. If this file is found in the current folder, then Flyway will use configuration parameters from this file.
Note, the inline parameters should override those in the conf file.
ConfigurationI find it much easier to use the configuration files. While you can specify this with the configFiles parameter, I often ensure that I run Flyway from within a particular folder that has a flyway.conf file. This has worked well.
Copy your flyway.conf from your install folder (under conf) to a new folder where you will experiment. For me, I set up a new folder called “smoketests” where I’m doing a little experimenting. As you can see, I only have two files in here:
The default conf file can get a little confusing. It’s full of many options, which are somewhat documented. My default looks like this when I open it:
Most everything is commented out, which makes it harder to figure out what to do.
The thing to remember is that you uncomment those settings that you want to set. For example, the drivers section is extensive. Some of theses are included, some you need to download. Since this are all Java based JDBC drivers, it can look strange to a SQL Server person.
The thing that gets lost in here for me is that I don’t comment out the line that starts with SQL Server. Instead, I need to copy and paste the jdbc part to the flyway.url= line. My valid connection would look like this:
Let’s try connecting with the info command. When I run Flyway info, I get asked for a user and password. Annoying, but not that bad. However, I then get an error:
This is a secure connection error. I do have TCP/IP enabled, which I know Java needs. This is more about a change the SQL Server team made, which requires secure connections.
Let’s add this to our connection string: ;trustServerCertificate=true
This gives me this string:
flyway.url=jdbc:sqlserver://aristotle\SQL2022;databaseName=FWPOC_1_Dev;trustServerCertificate=true
Now when I connect, things work. I still have to enter a name and password, but I can connect and I get back something.
I see some info on flyway versions. There’s a new version available. I’m also licensed for Flyway Enterprise.
Then I see the filesystem folder, sql, is missing. This is where the migrations are stored, so if I want to add migration scripts, I need to add a folder.
Next, I get the complete connection string. Lots of options here, of which most are defaults.
Then I see the schema is empty. Flyway works off schemas, which is what many platforms require. We get lazy in SQL Server and often just use dbo, but most other platforms want schemas set up.
Finally I see that there are no migrations run in this database, and I see an empty status table.
That’s a lot, and I connected to the database. Nothing changed in the database, and no flyway_schema_history table was added. This essentially let me know Flyway was working.
The last thing I’ll do is add this to my connection string: integratedSecurity=true
This lets get away from the name and password on Windows, but using Windows Auth for SQL Server.
That’s it. I’ll keep working on different commands in Flyway and getting to know the CLI.
The pandemic forced many events online, which has worked fairly well inside many companies and certainly spurred some technological advances in how we meet and share information. With the pandemic ending or at least entering an endemic phase, there are many live events now, as well as some attempting to embrace a hybrid philosophy.
I went to two conferences in 2022 that embraced the hybrid concept: SQL Bits and the PASS Data Community Summit. I was just a speaker at the former, and mostly a speaker at the latter, though I saw quite a bit of the behind-the-scenes action from the organizers’ perspective at the Data Community Summit.
During SQL Bits, I watched a hybrid session with a remote speaker. I also presented two sessions with a moderator that was engaged with the virtual attendees. The experience watching a remote presenter wasn’t great, and the audio at times was hard to hear. I also spoke with a few virtual presenters who also noted they struggled to hear people in the room. All in all, it felt somewhat live, but I felt disengaged from the speaker, and missed a feeling of interaction from the picture-in-picture of the individual speaker inside the larger screen. It was just too hard to see them and understand body language.
At both events, the sessions I presented with a moderator went well, and I felt the moderator could be a good proxy for people online. I also know that I have little visibility into what people online see and hear. I’ve gone out of camera range at times, and I rarely know if the virtual audience can see my entire screen, me, or some combination of both, much less hear what I’m saying.
After the Summit, I read a post about hybrid events from Andy Galbraith, which made a lot of sense to me. I enjoyed seeing and speaking with Andy in Seattle, though I know that our interactions would have been fewer, less satisfying, and less connected if we were virtual using Spatial.IO or some other technology. Spatial has been the best online platform I’ve seen for groups, but it’s still not great.
Like Andy, I don’t want to pick a fight, but I have a few perspectives on why I don’t think hybrid works for the most part. First, it’s resource expensive. The equipment, people, and time spent putting the show together make it less attractive for most events, even paid. The people managing AV find it more than twice as hard to manage the technology and don’t like the added stress.
Second, the interaction is still very limited, and when we do try to interact, it’s full of pauses and stutters in conversation that make this much worse than in person. Even a proxy adds delays and a very un-natural feeling to the engagement.
Lastly, we don’t really create the same engagement as the view from a virtual event is very limited, and one can’t really engage with other attendees in the same way we can in person. Not only in sessions but before/after and in other spaces. Meetings with more than ten people are often hard to engage with more than a few people, so I don’t expect we can make things better when there are 50, 100, or more people in a virtual meeting.
I appreciate the accessibility aspect of virtual attendance. I think we ought to have more virtual events. I was glad we had a few virtual SQL Saturdays in 2022 and I’d like to see more in 2023. If you want to put one on, especially with a niche technology, please reach out. I’d love to see more of these events.
I just don’t think hybrid works. Not at large conferences, not at SQL Saturday, and not at user groups. My view is you should choose one or the other. Make it in-person or virtual.
That being said, I do like the broadcast efforts from large conferences. I thought PASSTV was great. I have enjoyed the shows between sessions at Ignite and Build, and I hope we can do some of that for the 2023 PASS Data Community Summit. One thing I think has worked well across decades is broadcasting. The effort is lower and the experience for the remote audience can be better. They aren’t included, but they can see a “show” that is better than a clumsy attempt at bringing thousands of people into a “room” of some sort.
I don’t think hybrid conferences work, but I know I’m biased in that I help run and organize live events. I want to see more live events in the future, and I want to see more virtual events.
Just not together.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Next week, on Jan 18, 2023, I’m doing another webinar with the SQL Solutions Group. This is the SeQueL to our first webinar on Database DevOps.
You can register here.
Since the first webinar, Scott has been working with clients, integrating DevOps into their software process. It’s a big change in some ways, small in others. We decided to do another webinar, discussing some of the challenges clients face in moving to a DevOps style of software development.
Join me on Wed, Jan 18. Be sure you register for the event and set a reminder.
Today’s coping tip is to do a kind act for someone else today to brighten their day
This is one I’ve done before, but this tip reminded me to do it again.
My wife works hard, often lots of hours, and gets stuck outside with many clients. I made it a point this week to look at her schedule and then cook lunch for her and bring it out.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to make time to do something kind for yourself.
Learning to better take care of myself is something that I’ve been working on throughout the pandemic and continuing as we move to an endemic.
Last year I had ankle surgery, and it went well, but I stopped progressing in my recovery. I’ve struggled with balance and a little pain the last few months. I decided to take some time to work on this and try to get my body more healthy for the long term.
January is PT month for me. I’ve scheduled some physical therapy sessions that will help (hopefully) strengthen and improve my ankle. Taking a couple sessions a week, as well as daily exercises, to work on this for myself.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Not that I’m looking to do this, but that’s the invitation from Raul Gonzalez this month.
This is the monthly blog party where we write on a topic chosen by the host. All topics are tracked at T-SQL Tuesday, and if you want to host, you need a blog and then ping me.
Just Do a LittleI’m going to tackle two sides to this, the administrative/sysadmin side and the development side. I think it’s easy to implement worst practices, especially in busy environments, if you haven’t gained some knowledge and prepared yourself to do better.
The main thing I’ll point out is that it can be hard to implement best practices, or sometimes even decide what is a best practice. However, you ought to be able to avoid the worst practices.
Worst DBA Practices – Poor SetupOne of the things that has often been done in technology, especially in the Microsoft-based world, is accept defaults, get software up and running, and forget to tackle ongoing practices. In particular, there are two things that I consider worst practices: backups and sysadmin.
First, no backups. Above all, even above security, we need backups for our data. If we have those, at least we can recover. When you set up a new database, you ought to ensure you have backups implemented. Right away. I’m saddened that Microsoft hasn’t made it easy to implement this as a part of setup. While you can use unattended setup or dbatools or something, it takes a little prep.
At the very least, schedule Ola’s backup solution in each instance that is set up. At least with USER_DATABASES set, this will pick up new dbs as a backup.
Second, don’t use sa/sysadmin or any privileged account for applications or even DBA scripts. Set up another account that can be disabled, password changed, or some other security measure. Too often people never set up another account and get used to using sa.
Truly a worst practice.
Worst Developer Practices – Starting with SELECT * and NOLOCKAaron Bertrand has a number of bad habits posts, which I think are worth reading. If you can’t adopt his best practices, at least avoid the other issues.
Two worst practices I think create technical debt and later problems are SELECT * and NOLOCK. If you can’t do anything else, at least avoid these.
The first (SELECT *) leads to issues with extra data movement across the wire, extra reads in SQL Server, and in general problems with refactoring items as you never know where an application requires certain columns. I know we won’t get perfect, but don’t use SELECT * in any production code. The only place for this is when you want a SELECT TOP 10 * to get a feel for what data is in the table. Every other query in an application ought to specify what columns it needs.
Note: Using SQL Prompt and getting all columns is just as bad. Pick the ones you need.
Secondly, NOLOCK should not be a default item. There are data integrity issues, which can cause you problems down the road. Putting this in often means everyone is terrified of removing it. Don’t start here.
This post looks at how to set up a PostgreSQL container on Windows using Docker for Windows. I’ve seen a few posts, but I had to cobble together some instructions from places, so I decided to make my own post to help me remember and keep things simple.
tl;dr: do this:
- Create a folder c:\docker\pgdev
- get the Docker image: docker pull postgresql:latest
- Run the container, command below:
docker run --name pgdev -e POSTGRES\_PASSWORD=Str0ngP@ssword -d -p 5432:5432 -v C:\Docker\pgdev:/var/lib/postgresql/data postgres
That’s it. Then you have a PostgreSQL instance running on port 5432 (default) with a user, postgresql, and a password, Str0ngP@ssword.
More detailed instructions below
Create a place for dataContainers are ephemeral, which isn’t what we want for a database. We want to keep data around, so let’s make a place for this. This will be a volume for our container, which we will map to a particular location inside the container. Then if the container dies, we can map this to another PostgreSQL container and have our data appear.
Create a c:\Docker folder on your machine. This is a good spot for any Docker related volumes.
Now create a pgdev folder under c:\docker. This is the place we’ll keep data for this PostgreSQL container. It should be empty.
Get the ImageContainer images are available from Docker. I won’t cover installing Docker or setting up Linux containers, but you do need to do this. I used Windows 10, and I have WSL v2, as you can see:
Docker Desktop is running Linux containers. You can see that since it say “Switch to Windows containers”.
Next, use Docker Pull. I assume you are working with the default Docker registry, so this command should work:
Docker image pull postgres:latest
This will start downloading the image.
When this completes, move on.
Starting the containerOnce we have the image, we can start the container. You could do this without the folder above, but your data would be in the container and if the container were ever deleted, then the data is lost.
The basic command for starting the container requires a few parameters. Here is a list of what I provided in the command at the beginning:
- –name – This is a name you can use in docker commands to refer to the container. You can put anything. I chose “pgdev”.
- The password in the database system for the postgres user. This is a default user and you send this in as an environment variable with -e. The password I used here is: Str0ngP@ssword
- -d runs this detached, rather than interactively. This means your command shell can return command to you. Otherwise, all output from the container appears in the shell and you can’t type anything.
- -p is the port mapping. This is host:container. In this case, we map 5432 on the host (where we use some postgreSQL driver to connect) to 5432 inside the container. You can choose any unused port for the first number, but 5432 is needed for the second number as the postgresql service is listening on 5432.
- -v is the volume mapping. Here we map our host folder to a container folder (host:container). We enter the folder we created above and then map this to the place where postgresql stores data. That’s in /var/lib/postrgresql/data
- postgres is the image name.
Here is the command again:
docker run --name pgdev -e POSTGRES\_PASSWORD=Str0ngP@ssword -d -p 5432:5432 -v C:\Docker\pgdev:/var/lib/postgresql/data postgres
Once this executes, we should see a long hex code returned, which is the container identifier.
We also see our folder is now filled with postgresql specific data files and folders:
That’s it, our container is running.
Today’s coping tip is to look back at a previous coping tip that required planning and evaluate how it helped.
One of my tips in December was to listen to new music. I downloaded the four sets of music for my trip and listened to them. I was surprised how I felt.
- This is Santana – A good mix, which I enjoyed. #2 on this list. I listened a few times to this playlist.
- Blessings and Miracles – newer music, but not as enjoyable as I thought. I listened once and then part of a second time and gave up. #4
- The Pinkprint – New, and I’ve enjoyed some Nikki Minaj on other playlists. However, I didn’t enjoy this one apart from a couple songs. I did listen a couple times, but found myself drawn to the other album. #3.
- Beam Me Up Scotty – I found myself listening to this a number of times, maybe 5-6 between my airline trips to London and Lisbon. #1 on this list.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Sometime during the first 3-4 months of the pandemic, after offices all over the world closed, I thought that a lot of businesses would try to hold on to existing employees and minimize turnover. I also thought many employees would be nervous about changing jobs amid all the uncertainty.
That wasn’t the case everywhere, and I was surprised at how much employee turnover occurred in many companies during 2020 and 2021. We had the Great Resignation, where talented employees (and some not-so-talented) found new employment with lots of remove flexibility. We typically have low turnover at Redgate, but more surprising to me was the number of people we’ve hired in the last two years.
Our gains come from losses elsewhere. It seems a lot of companies have had a bit of turnover, more than they liked. The costs of replacing a staffer has climbed to an average of USD$57k, according to a recent poll of hiring managers. I’m sure this is a combination of the time and money spent on recruiters and interviews as well the lost productivity of having less people around.
For many of us working in technology, we realize that losing good people is a problem. Often we lose undocumented knowledge about our systems, those shortcuts, checks, small tasks, not-so-obvious fixes, and more that make life easier. We can feel more stressed about additional work, or even the anticipated unknown challenges we might face. Moral, productivity, and motivation decrease, which creates a snowball effect. Everyone gets less done, which costs the organization more. Either lower revenue, more costs, or perhaps fewer services for those government or non-profit concerns.
With large tech companies letting lots of talented people go, I wonder if we’ll see more competition in the employment market. For quite a few years there have been fewer workers than there are jobs. Perhaps that changes a bit in the future, though I still think there are not nearly enough talented workers for the positions open. Good for talented workers as they may have more choices, but also good for less talented workers when companies feel pressure to hire anyone.
Except, what I’ve seen in a lot of companies is that they don’t want to hire anyone. Having a degree or cert isn’t enough. Companies aren’t going to hire you because you are Azure/AWS/etc. certified. They want tech skills and at least a few soft ones. That creates an even higher cost to fill positions, so maybe that USD$57k isn’t a bad number.
To me, what this really means is that organizations ought to look at poor management, both middle and upper, and replace those people. That’s the problem in many places, one that isn’t easily fixed. Poor culture, micromanagement, lack of support, little-to-no psychological safety, and more contribute to poor retention.
Culture is important and building it is hard. While not everyone will have a great culture, many organizations can avoid a poor one by treating people fairly and ensuring management understands how to do that. If for no other reason than to avoid the high costs of replacing staff.
Steve Jones
This is the goal setting post for 2023. The previous ones were:
- 2022
- 2021
- 2020
- 2019
- 2018
As with previous years, I’ll break these into three categories: career, personal, and community. There’s some blend with all of these, but these are goals that I want to use to engage and grow my life in interesting ways. To aim for mastery, autonomy, and purpose, as Daniel Pink wrote in Drive.
The idea here is to set goals for Q1 2023 and review in March. I’ll also do monthly reviews of these. The rest of this post should be repeated in all future posts.
ReadingI’m separating this out, as I’ll put a few things in here that span other areas.
For Q1, I’m aiming for 2 books outside of the fiction I enjoy. These are going to be in these areas:
- marketing – title TBD. Something to help me better understand how to approach this part of my job. There have been recommendations in our Slack channels, so I’ll pick one.
- Coaching – Again, a few recommendations in groups I frequent.This is important as I influence young people.
Both of these are items I’ll target for these reasons:
- S – This is targeted, reading a book
- M – I’ll finish or not
- A- both
- B – 1 done, 1 partial
- C – 1 done
- D – partial
-
F – not started
-
A – I read around 100 books a year, so 2 in a quarter isn’t too much.
- R – Marketing is a part of my job and career at this point, so this is good. Coaching is something I do personally with kids and with customers for DevOps.
- T – I have 3 months, until 31 Mar.
CareerMy career has taken a strange turn in that I continue to do some technical work and consulting with clients and customers, but more of my effort is spent understanding marketing and sales. I pivoted slightly last year by reading The Trusted Sales Advisor,
I also need to do more work with other platforms, at least to become more comfortable with them. I thought I might do more in 2022, but ended up getting buried with community stuff.
For Q1, I’m doing to aim to do a couple things:
- Set up a tracking system based on the book for my efforts with customers
- Track and calculate my scores to help me better approach how I work with customers.
- Review this with my boss and with someone in sales
This is not a lot, but it’s a good start. I can review this goal as:
- S – This will include a way to track things, likely Excel, and then some formulas or reporting to help me see how I’m doing. I also need to fill this out after each call with a customer. I’ll send it to a couple people in sales for review in March from two people. setting a reminder now.
- M – I will have a sheet to look at, and a record of calls. I also have a review reminder set.
- A – All calls tracked, review complete, data used to change my work.
- B – most calls tracked, review complete,
- C – half calls tracked, only 1/2 reviews,
- D – less than half calls tracked, no review,
-
F – no sheet set up
-
A – Building a simple tracking sheet ought to be easy. The formulas are given. We used to track calls, now I’m adding in 5 minutes to think about how things went. Getting people to meet with me should be achievable if I make the time.
- R – Important for my job
- T – I have a 31 Mar deadline.
CommunityA portion of my job is community. It’s also a passion. I spend time here outside of work, I volunteer and go to UGs/events when I don’t have to, and I certainly do work with SQL Saturday outside of my job. In fact, I was fixing PRs and updating the site while on vacation in Portugal, mildly anoying my wife.
In terms of a goal here, I’m going to focus most on SQL Saturday and less on speaking for now. I likely will go to a few events, but I want to limit my efforts in Q1.
The goals here will be the following:
- 2 speaking engagements for community (SQL Sat Austin and SQL Bits are on my schedule)
- Reach out to 10 SQL Sat groups that did not run an event in 2022 and motivate them for 2023. This should be at least 2 emails to each person/group.
- Reach out to all 2022 organizers and ask about plans for 2023
- Start coding a tool for converting schedules to HTML.
- Hold office hours once in Q1
- Send 3 monthly SQL Saturday updates to the community
In terms of the review, I see them this way.
- S – These are fairly specific. I can track emails. I can’t quite track motivation, but that’s something I need to do to help grow SQL Saturday back to where it was. I also can start putting code in a repo, which is a big step.
- M – For the review, I’ll say this:
- A – 2 presentations complete, 10 emails to new organizers, 36 to older ones, at least 3 commits and a MVP of a tool, office hours, 3 updates
- B – 2 presentations complete, 8 emails to new organizers, 30 to old, 2 commits, office hours, 2 updates
- C – 2 presentations complete, 5-8 emails to new, 20 to old, 1 commit, office hours, 2 updates
- D – 1 presentation, 3 emails to new, 10 to old, no commits, no office hours, 1 update
-
F – lower than D
-
A – This can be tough. Finding time for SQL Saturday comes a1nd goes, as does motivation. I should easily do the talks, and I can easily reach the emails by making time. Office Hours can be tough, and the monthly newsletter is something I’ve struggled with.
- R – These are community efforts that should help others. They don’t directly help me, other than building some skills as a leader, but they will help others if I can get more events moving.
- T – All bound by the 31 Mar deadline.
PersonalMy personal goals are going to be a little shortchanged here. When I look at what’s above, that’s more than I probably should tackle, though some of them should be quick to get through with a short bit of time every week.
Coaching also ramps up, so I’m going to limit this to a couple things for Q1.
- Use my Power BI report for stats
- Update the Power BI report based on feedback from athletes
- Build 6 wooden coasters – I haven’t made enough time for my hobby here, so I’m going to start small. We need more coasters (as noticed over the holidays), so I want to build 6. Same style, different.
These can be rated as follows:
- S – Use means I gather stats. Updates are from feedback. Coasters are specific.
- M – Have I entered stats on the sheet, did I respond to feedback, coasters.
- A – All stats entered in sheet/report. Take all feedback and do something, 6 coasters
- B – all stats entered, some feedback, 4 coasters
- C – All stats entered, feedback not influencing report, 2 coasters
- D – some stats entered, no feedback used, no coasters
-
F – failing at stats
-
A – These should be achievable. The coasters should be 2 short periods
- R – The first two are relevant to my efforts coaching. The last is just for satisfaction and to remind me of hobbies.
- T – 31 Mar.
Last year I set a bunch of goals, and I thought I did pretty good working on them. Not amazing, but pretty good. This year, I want to attack goals a little differently, so I’m going to think about how to measure things first, then set up the goals.
One system of looking at goals is SMART. This stands for:
- specific
- measurable
- achievable
- relevant
- time-bound
In terms of my goals, I want to tackle some in this format in 2023, and see how this goes. With that in mind, I’m looking at things in this way. I have a few sample goals, but I need to think through these a little more.
SpecificIn looking at 2022 goals, they were fairly specific, but not always. I didn’t define the personal ones well. The work and community ones were good.
In 2023, I’ll look to ensure I specify each of these in more detail and have something to measure against. I need to use the 6ws (who, what when, where, which, why) where possible.
MeasurableThis is the big one. Do I have metrics by which I can measure things. Certainly percentage read for books, but what about my demos? Can I really get these to a percentage of done? Or just work/not working? What about the number of events? That’s easy, but does it include other metrics. I ought to ensure I try to put some sort of way to definitively note whether I’ve achieved something or not.
Examples could be:
- Send 12 SQL Saturday newsletters
- Reliable working process that moves data from source to target with more success than failure
- Volunteer 4 days
Achievable This is interesting. I thought all my goals were achievable, but not easily. If nothing new came up or life didn’t get in the way, I could achieve those. I think I’m doing well here, despite not finishing everything.
RelevantMy career is in a weird place. I’m closer to the end, and my growth in Redgate is more focused on marketing and sales, so less on technology. The tech matters, as I consult with customers and train them, but I do a lot more things that are very different.
At the same time, all of the items I chose last year do help my career. Whether this is for content, for knowledge I’ll take to customers, for the recharging from volunteer days or reading. I don’t usually have a problem here.
Time-BoundThis is harder. No matter how I’ve planned, usually things get away from my and a portion of my schedule isn’t my own. I get asked to do things or assigned them, and that messes up plans.
Life outside of work can also get in the way at times, and leave me less time to work on my goals.
I’m going to tackle goals a little differently this year. I’m making them quarterly, thinking that’s a more manageable time-frame where I can predict some level of commitment.
We’ll see.
Aims for 2022For this year’s goals, I’ll try to work within the SMART system for goals. The T here, will be aiming for quarterly re-alignment of goals with life.
I’ll still rate myself monthly and then roll those up for a quarterly review. I’ll adjust the goals quarterly as my schedule reveals itself. In terms of ratings, I want to use a school-type rating of A-F, but with these views:
- A – exceeding expectations for goal achievement, making almost all goals
- B – above average for goal achievement, making much more than missing
- C – average goal achievement, making some, missing some
- D – below average, missing more than making
- F – completely missing most
While I was mostly an A student in high school, with a good set of goals, I ought to be a B-C student. Life will get in the way, things will change, and I shouldn’t expect that I can devote as much time across the next few months as I plan for during this slow time of year.
There was an article on the worst technology of 2022 from the MIT Technology Review. I was hoping to get some sense of what was really useful or failure-prone technology, but it seems that the article delves into decisions on how to run technology or policy more than the actual technology.
The list is here, with my short comments on those that might apply as actual technology items:
- The FTX meltdown – no
- The fentanyl crisis – no
- a pig heart to human transplant – maybe. A biotech attempt that didn’t work
- the issues with zero covid in China – policy, not tech
- Twitter rules – no
- Ticketmaster system meltdown – this counts
- Meta’s Galactica – definitely among the worst
Many of these seem more about decisions made in how a platform or technology is used, rather than the merits of the system. The FTX thing was less even technology and more a scam using software that likely worked as intended.
For the best technology, I find many lists that delve into gadgets. Those can be fun and certainly might feel like they change how you use technology. From that list, I know that mobiles and headphones can be exciting, but these always feel incremental to me. I actually have preferred seeing how cheap mid-range headphones have become. I do think the rings measuring health instead of a watch-like device is a little innovative. Tablets, laptops, and TVs, meh.
In terms of innovative tech, this is quite a list. Shooting a probe at an asteroid is amazing engineering. A personalized info board, seen differently by many different people is fascinating, though a bit creepy. The DALL-E 2 image generator was certainly incredibly tech, albeit controversial. The 3D-printed ear is amazing as well.
The world of technology for many of us (frameworks, languages, software, hardware) changes so fast. Vendors have shrunk their lifecycles as they move to a DevOps style of working, which creates pressure on us to keep up, or at least be able to learn quickly as versions change. We certainly don’t want to work on the worst technology, and maybe not the best because it changes fast. Instead, we need to pick and choose those items that are relevant to those that hire us to get things done.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to do something outdoors today. Enjoy nature.
I’m writing this a few days ahead, so I’m using an activity I did over the weekend. The weather was somewhat mild, though still snow around. However, my wife and I went out for a walk with the dogs, taking some time to get outside, slightly bundled up, but enjoying being in nature with each other and our dogs, a relaxing, calm exercise day.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to set aside a regular time for the next month for an activity you love .
This is a big month for me, with coaching continuing. That will eat up a lot of time. However, that does mean that I need to focus to ensure I continue to enjoy life.
I’m going to make an effort to spend at least 10 minutes 5 nights a week on guitar. I enjoy learning and playing, and rather than defaulting to reading or media right away at night, I’m going to keep the guitar near the bed and ensure I spend a bit of time playing some songs.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
One of the things I’ve been trying to do is dig in more deeply to the Flyway command line (CLI) as part of my work with Redgate. While Flyway Desktop is amazing, it’s a wrapper, and my inclination is to ensure I understand the underlying technology.
When I first downloaded Flyway, it wasn’t obvious what needed to be done to connect in different ways, so I decided on a short tutorial that would help me remember how and teach others a few things. This post goes over downloading Flyway and getting started.
SetupThe setup for Flyway is easy. The Community version is free, though there are also Teams and Enterprise editions.
Flyway Community works on many platforms. I’ll choose Windows below.
The Windows version is a zip file. Click this, and it will download.
Then unzip this. I have a c:\Utilities folder where I keep stuff and I’ve added a flyway folder there. I unzip the latest into this folder.
If you care about versioning, you might drop Flyway into a version named folder, but I typically don’t on this machine. For production deployments, I would be more careful.
You then need to add this to your PATH. If you are on Windows 10 or 11, then in the Control Panel (Windows+I), search for environment. Edit the variables for your account.
Once the new dialog appears, find the PATH variable and click Edit.
Now add an entry for the folder where you extracted the Flyway files. For me, this is c:\utilities\flyway.
That’s it, Flyway is installed.
We can check at the command line with “flyway version”.
If you have entered a license key, then you’ll see that message.
The world of technology has undergone an amazing revolution in my lifetime. Things that I thought were incredible in my youth have come and gone in the world. I’m sure that there were dramatic changes in technology across lots of other generations, but I wonder if they were as dramatic for others as they feel for me.
As I watch older movies, I’m amazed at the evolutions. I was watching an older sitcom recently and saw someone using a cordless phone. While we have these cordless mobile devices, there was a time when having a phone in my house that wasn’t tethered to a spot on the wall was incredible. And I haven’t seen one of these in years.
My Mother worked in real estate and getting an answering machine was critical technology for growing her business. These were essential in every home for a long time (along with call waiting). First tape-based, then SSD, but now these devices are really gone and exist in the cloud. Now you get voice mail and call waiting on a mobile device. Does anyone have an answering machine anymore? For that matter, are there still public phones? I’ve seen a few booths in my travels, but they seem to be mostly standing as tourist attractions for pictures or repurposed as small libraries or wi-fi hotspots.
One of my big career steps was working with an import company that lived off faxes. When I was there, we went from a single computer-based fax service to upgraded multi-line fax add-in boards with not only digital fax reception but the OCR of the actual recipient to email the images to the appropriate department. It was an amazing job where I learned a lot about imaging and large-scale storage of data. I know there are companies that still have fax machines, but emailing images around seems to have overtaken this in much of the world.
We still have mainframes and lots of older applications, but one piece of software I expected to see around was Lotus Notes. I know some people might still use it, but IBM doesn’t support it. They’ve outsourced. I guess a lot of software I used early in my career is gone: Ami Pro, WordPerfect, Lotus 1-2-3, and more.
Maybe one interesting device my wife loved was a Palm Pilot. There were lots of PDAs for a while, but the advances in mobile phones completely did away with that category of devices. I’m actually still amazed that Blackberry couldn’t survive the transition to screen-based devices. For a time it felt like everyone in business carried one around.
Hard disks and tape changers seem to be fading, though I’m sure someone still uses them. Anyone bought a tape drive for their personal or business use anytime in the last few years? Or for that matter, do you still use CDs or floppy disks anymore? VHS tapes? Dot-matrix printers?
What about a modem? The sound of a modem connecting was both amazing for me as a young man and an annoying sound early in my career. Same for pagers. I loved getting one but hated carrying it. While there might be these devices in use somewhere, for the most part this technology has been removed from our lives.
Perhaps one of my favorite devices, which came late and died quickly, was the iPod. I had a few digital music players, but they were hard to manage. The iPod was revolutionary, and I had quite a few over the years. I still have a waterproof Shuffle I use when I swim, but for the most part, these devices are just memories.
What tech do you miss, or remember fondly? Or perhaps, what old tech do you still work with that you enjoy?
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to share something helpful with a friend that you learned recently.
I share a lot with my wife and kids. They are a big part of my life, and it’s important to make them a part of mine. I’m not a big shar-er in general, and they might say I don’t share a lot, but they get more than anyone else.
One of the things I learned recently was during my volleyball coaching certification. One of the modules looked at a seasonal plan for coaching. I hadn’t thought of breaking the season this way, but the advice was:
- First third – 50% single skills, 30% controlled drills, 20% play, bonding
- Second third – team system, look at stats, strategies, hide weakness
- Final third – 80% play, offense/defense concepts
I think that’s a good approach and my wife and I are thinking to implement this.
The other thing I learned and shared was serving percentages over time from different locations to different ones. We’re emphasizing players think about this:
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to use one of your strengths in a new or creative way.
I have a lot of strengths. They’ve helped me to be successful. This tip is a hard one for me, as I need to get out of my comfort zone and try something new.
One of the strengths that has helped me is explaining concepts, ideas, etc. to others. It’s a lot of what has helped grow SQL Server Central over the year.
I’m going to take some time to explain how I approached some problems in Power BI and then take feedback. I’m sure there will be lots of suggestions and advice that helps me improve, but my goal is to provide a model that even though I’m not a guru or an expert, I’m going to display what I know, and then improve based on what I learn from others.
Hopefully I get more of you to blog and showcase your knowledge in the same way.
I’m starting with two projects. A Flyway one (first post tomorrow) and a Power BI one next week.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to start the new year off with something new – listen, watch, read something completely different.
I started listening to more country music last year. It’s an interesting genre, especially from the guitar standpoint. I’ve learned a few songs and want to work on a few more.
I’m starting today out with a few playlists that are outside of the normal things I listen to. No work (maybe), but I’ll put in the top songs from
- Kelsea Ballerini
- Luke Combs
- Thomas Rhett
It will be an interesting day.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
We had an incredibly cold front move through Denver last week. It was 46F one early afternoon, which dropped to 0F by the evening and down to around –15F overnight. Crazy cold, with snow.
Driving the Tesla was interesting.
This is part of a series that covers my experience with a Tesla Model Y.
Driving to TownI went to the gym the next day, and saw this as I headed out of the neighborhood:
It’s not a great photo, but what I noticed was that the 43mph had the D above it, for Drive, but not steering wheel for Autopilot. I use it on this road at times when traffic is light. You can somewhat see the road is a bit visible, and you can see some of the yellow through the snow in the middle, but no right line.
The cameras can’t tell where the road is, so they won’t let the car drive. I’m not sure LIDAR or RADAR, or anything would work here.
Here’s another shot in town:
If you look past the steering wheel, you can see the road looks better. Not great, but better. There are curbs here, so radar/lidar might work. Certainly humans can see them, but the car again doesn’t have the steering wheel showing Autopilot is available.
Update 1:
One other interesting note, a few days later, I was driving this same road, but more snow had fallen against the center curb. In essence, the middle lane was narrower because of snow piled up on the left side (in the image above).
I got repeated “lane departure” warnings, well 2-3 in a 1/2 mile, as the car thought I was crossing the lane boundary on the left. Strange, and disconcerting.
Update 2:
The next day as snow was melting, but the roads were very dirty, the car struggled to find the middle dividing line. I could see it as a human, but it was dim and barely visible. The car swerved a few times as it couldn’t always see where there was a turn lane on either side (L or R).
Can Level 5 Work?This is a great example, to me, of why Level 5 is not likely to occur for cars. Maybe Level 4 within certain conditions and in well mapped/known locations. Or perhaps with sensors in roads that cars read. However, a little snow (this was 2-3 inches) has blinded the car.
Humans drive on these roads all the time in snow. To be fair, many drive poorly, and often they don’t know where the lines are. However, we negotiate. We slide our car slightly left or right when there is other traffic. We aren’t perfectly in the lanes, and depending on the plows, we might end up “losing” a lane and just working with one path that isn’t really in either lane.
Can computing learn to do this? Maybe, but it’s feels like an even more complex problem, especially when you now are asking the computer to react to how other drivers are moving on the road. Negotiating a route with them. Can a computer see another driver wave them on or ask them to stop?
Tesla isn’t Level 5 or anywhere close. It’s Level 2, and perhaps Level 3 with the beta self-driving software (which I haven’t seen). I don’t know that I think anyone is closer, though I’ve heard the Google/Waymo taxis in Phoenix are pretty good. I’d like to try one and see, but those aren’t Level 5.
I’m not sure I would trust Level 5 if someone claimed it, at least not unless every other vehicle was also a L5 controlled car.
I do enjoy Autopilot and it was worked well for me. The FSD version was OK, but not enough better for me to subscribe to it. I look forward to cars becoming better and safer, but I’m not expecting to be able to go to sleep in the car and have it take me to my destination anywhere, anytime. I just don’t think L5 is anywhere within reach.
This is the last workday of 2022. Next week starts a new year, and as I’ve often done, I wanted to look back at the year. This time I decided to look back month by month, at some of the headlines and memorable data-related topics. I’m tackling things month-by-month.
In January there was a set of “tech experts” who shared their thoughts on the best database management systems. This one is worth a read for the humor involved. I wouldn’t really consider many of these to be DBMSes. I thought about including this one as an April 1 joke, but it was a real story. I get asked for my opinion at times by writers researching a topic they don’t understand. I hope I don’t come across like a few of these people.
February was another sad data breach story. In this case, from the state of Washington where many tech people live and have startups. It wasn’t clear initially what happened, but later articles noted this was from a stolen device. To me, this was a great reminder why dev machines (and databases) should NOT have PII. Mask/obfuscate/anonymize that data please. Or at least delete my name from your dev systems.
March had another humorous story (to me): Oracle is going to lure people away from AWS and SAP with their new offering. I could believe the latter, but not the former. Oracle hasn’t ever been good about pricing and it seems more people are leaving Oracle than coming to it.
April is the month of April Fools, but this isn’t a joke. Another data breach, again from a dev system. This one from Fox News, which included information of not only employees, but guests and celebrities. The claim that this was a dev system and not production doesn’t matter if the data is real. Please people, keep prod PII out of dev.
May had a funny post from Hacker News. Someone put their whole life in a database. The comment that caught my eye on HackerNews: “Men will literally devote hundreds of hours to building a bespoke database tracking every moment of their lives instead of going to therapy.”
I worked a lot on my weight and diet in 2022. June had me finding a public database to help me choose better food. A public database on processed foods. Great idea, but everyone has an agenda. I hope this has some crowdsourcing and reasoning and isn’t just one person’s opinion.
In July, another data breach. This time in China with information for 1 billion people. Wow. I dislike large databases for this reason. It’s also a good reminder why you ought to remove information from your databases over time, at least the PII part. Again, delete my name, if nothing else.
There are so many types of database platforms. Have you heard of a vector database? Apparently, it’s for managing vector embeddings, whatever those are. An August article on the strange growth of the database market.
September started this crazy AI art craze. There was a call to remove living artists from the database of works that an AI uses. Makes sense to me. I think artists deserve support and while I like AI doing new things, maybe wait until the artist isn’t producing work.
October showed a good reason why we need ongoing patches or open-sourcing of code for retired systems. There was a 22-year-old vulnerability reported in SQLite.
In November, what other news could there be than Lego Steve? It was a tiring week.
December is just ending, but I’m ending on a reason why databases without auditing are a problem. Men behaving badly in this one. Gathering public information (or even semi-public) at scale can be problematic, and the information gets abused. Better controls, but also more auditing and triggering of some actions to prevent this (and other) sort of abuse.
Let me know if you remember these events, or perhaps if there’s a favorite memory of the 2022 data world that you wish I’d included.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I set goals at the beginning of the year, and I’m tracking my progress in these updates during 2022.
December is usually quiet, though this year I spent the first 12 days of the month out of town. That meant that I wasn’t getting much done, either at work, in my life, or on my goals. I did a little reading, but mostly it was work and holiday.
I’m going to do the grade for the month and then look at how I feel about the year.
Grades:
- December – B+
- 2022 overall – B
I made some good progress in December and managed to get a bunch done on my report, finished a book, and started on my demos again. Not amazing, but really good.
For the year, I think I’ll give myself a B as I managed to read all the books, do some demo work, and get my Power BI skills growing. I also did a bunch of community stuff, though not quite as I saw it happening at the beginning of the year. Still, a good year overall.
Here are the goals and updates for the month.
WorkMostly reading this month, as I was traveling. I did reset my environment for a DevOps demo and got projects set up for SQL Server and PostgreSQL in the same repo.
- DP-900 – Passed
- DP-300 – Stated studying, practice tests around 80% correct, need to schedule
- Demo with SQL Server – 20% – dev db set up, started on pipeline
- Demo with PostgreSQL – 20% – Dev db set up
- Demo with MySQL – 0%
- Thanks for the Feedback – 100% – HIGHLY Recommended
- Good Strategy/Bad Strategy – 100% – VERY interesting
- Do Hard Things – 100% – HIGHLY RECOMMEND so far
- The Trusted Sales Advisor – 100%
PersonalSent my dashboard to a few others to use. Not the entire team, but I got 4/10 using it to see what they think, which is a good start. With my assistant, we’re at 50%
- Link Google Sheets to Power BI – 100%
- Create Report – 75%
- Create Dashboard – 60% – Added a main page
- Get people using it – 50%
- Coaching Certification – 100%
Community* Support the Colorado groups by speaking twice and helping getting two events set up
– 66%
+ CO Springs event set up and executed
+ Denver SQL Server SQL Saturday executed
- Speak at 3 other community events (was user groups) outside of Colorado – 100%
- Spoke at DBA Fundamentals group
- Spoke at Toronto SSUG
- Spoke at SQL Saturdays in Jacksonville, New Jersey, LA, Denver
-
Boston, Toronto in Oct
-
Support SQL Saturdays – Help get 10 events run in 2022 – 100%
-
- We have had 15 run this year and 3 more scheduled
- In Nov, we had 5 events complete – Sao Paulo, Oregon, Bangladesh
- Seven events scheduled for 2023
- Volunteer 4 days with Habitat – 0 days
- Not happy about this still. I had hoped to go Sept 15, but they cancelled the build that day. Need to find another day, but it’s hard. They aren’t doing every Wed/Thur/fri, and travel is getting in the way.
Today’s coping tip is to give thanks. List the kind things others have done for you.
I know that my life is great, but it’s because of a lot of help I get from others. I do a lot, but I can’t do it alone.
I thank my wife for supporting me and comforting me.
I thank my kids for the same, and for helping out around the house, which lets me tackle other things. Not the least of which is helping their Mom so I don’t have to do as much on the ranch.
I think Andy, my former business partner for the friendship, counsel, and reminders of things that I should think about, but sometimes don’t. He helps keep me balanced and thoughtful about the world.
I think my bosses at Redate this year, Tom and Annabel, for the support and help at work. Cecilia was wonderful in supporting me at events, and lots of others taught me things and helped me through the year. They helped me cope and thrive during a busy year.
I thank all the SQL Saturday organizers for their efforts during a tough year of restarting the brand. I think the speakers and helpers for volunteering and the attendees for coming. It was a year that exceeded expectations (blog). These people helped me with inspiration and confidence this was a good idea.
I thank my friends for the time, conversations, lunches, dinners, drinks, and wonderful times spent together, which fulfilled a lot of social needs.
I want to thank the kids on my volleyball team, both for the 2022 season and those I’ve started to work with for 2023. They give me hope for the future, they extend my family by being a set of surrogate daughters when my own is away from home most of the time (and not a child anymore), and they inspire me to work harder with their efforts.
There are plenty of other people to thank, but these are the top of my list.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to ask for help and let someone else discover the joy of giving.
I do most of the cooking in the house, but I thought with family around, I’d get some help. I usually don’t mind preparing, but a few times I asked others to help out and handle some tasks or prepare a portion of the meal. Little things, as I like cooking, but I also thought others would enjoy the chance to contribute.
Hard for me, but it was still fun to let others help. I’ll try to do this more often.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
We got a solar system in 2022 to try and fix some of our power costs and hedge against the future. We were fairly confident it would work well, but weren’t 100% sure. So far, it has reduced our power bill quite a bit, though it’s a bit hard to tell right now if this is actually saving much money.
However, the performance is above what was estimated. I’ve built a Power BI Report that tracks the actual production v the estimates. So fat, almost every month has beaten estimates and our power production is well above what we expected.
Given the fact that power prices have increased in CO and I expect more increases to come, this is nice.
I can’t embed the report here, but this is an image of the report:
I need to add some more costing here, but our current bill is often around $22/month, of which $13.50 is a connection charge. Usually we are paying around $8-12 of demand charges for peak usage from 4-8pm. Hard to get the family to slow down here, and there isn’t much solar during this time.
This is, however, much better than last Nov/Dec, when we had $220 bills. Even with the loan payment, I think we’re paying less than in the past.
This month’s T-SQL Tuesday invitation asked about end-of-year operations that you might take on your databases. I don’t know how many of you follow any specific end-of-year routine, but you may.
However, that got me thinking about an end-of-year career evaluation. It’s not something I’ve done, but I do often look back and forward and build some career goals. That has been helpful in ensuring I do some work on my career each year, even though I’m much closer to the end than the beginning.
If I were younger, I would look at my career in three ways: what I need to learn, what I want to accomplish, and how to move toward my goals. I’m actually going to tackle these three areas myself, but since we’re near the end of the year, these might be things you think about as well.
Technology changes so much, and we are often challenged by new requests, requirements, or circumstances to work with new software. I think the cloud has spurred this on, and as a result, I expect most people find they need to learn things constantly. Having a plan, and keeping a list of the technology is a good way to capture the entire bundle of challenges you face. Whenever someone asks you about something new, or you hear in a review that you should improve, add those things to your list. It’s likely you’ll never get to them all, and certainly, you won’t be an expert in them, but you can develop lots of competence and fluency in different areas.
The second thing to look at is how to take this big list of skills, languages, software, etc. and prioritize it. Cut the list down to something manageable this year. Or maybe just this quarter. Being organized can help you deal with the overwhelming number of items on your big list. In this stage, I’d order things by what’s interesting to you. This might be an understanding of YAML to update pipelines as code. It might be solving some types of complex queries in SQL that you struggle with. It might be learning to better express yourself clearly and succinctly in email. List the things you need to work on.
This isn’t easy, and this is really a prioritization of what is most important to you. You might talk with co-workers, your boss, or your partner to help decide what you can do. You should also think budget here, in terms of your time. Can you spend more than 8 hours a month on something? Carving out 2 hours a week is likely manageable, but more can be hard. Think hard about this, especially if your employer doesn’t give you any time while at work.
The last evaluation should be easier. It’s taking the things you are working on and building a plan to actually improve yourself. Do you need to draft better crafted emails and have people review them and give suggestions? Are you going to pick specific problems to solve in SQL? Maybe you want to tackle a specific Power BI report? All these are examples I’ve used myself, but you might have different things from your second list. This is the short-term plan you’ll implement.
I wouldn’t make this too detailed a plan, but I would include enough to guide you. Maybe this is actually spending a month of lunches on something? Or just two lunches a week? I wouldn’t schedule them all out, but I’d have some specific things I need to tackle, in some order, which at least gives me a plan. I wouldn’t include timelines here, but rather, what is the first thing to do, the second, and so on.
This sounds like work, and it is. That’s how you move forward, and how you shape your career to be what you want. Be deliberate, make decisions that are best for you, and then execute on those decisions. That will help you find the best career and job for you.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to look for something positive to say to everyone you speak to.
This works well during the holiday season as most people are in a good mood and share happy thoughts. Just before Christmas, as we ran a few errands before the holiday, I made it a point to do this. My efforts:
- Tell the person checking me in at the gym they looked nice
- Tell my yoga teacher that I appreciated her holding the class as it was welome
- Wish a friend in class happy holidays and ask about their family, hoping they enjoyed the time together
- Thank, tip, and complement the person making my coffee on their outfit after class.
- Wish all cashiers happy holidays.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I’m heading back to VS Live in March 2023 for the Las Vegas show. This time it’s at Planet Hollywood, which is a hotel I’ve never visited. I’m excited to go to a new place, and I’m looking forward to seeing lots of friends at the show.
If you want to join me at fun show, register with the code Jones for $500 off the price.
VS Live is a multi technology show. I always learn something interesting there from Brian Randall, Rocky Lhotka, Bob Ward, Andrew Brust or any of the other amazing speakers. There are lots of development sessions, both technical and soft skills, along with a few of us data people delivering talks on how to deal with the database and power your software.
The schedule is up and it looks exciting. I have sessions I’m delivering:
- Architecting Zero Downtime Database Deployments
- Adding Graph Structures to your SQL Server Database
There are lots of other great talks as well, so think about kicking off your 2023 development efforts by coming to VS Live Las Vegas, Mar 19-24, 2023.
Register with the code Jones for $500 off the price.
I’m doing another webinar on Jan 18, 2023 with the SQL Solutions Group.
You can register here and reserve a spot.
Scott Klein and I did a webinar last summer, where we talked about SQL Server and how it integrates into DevOps. Since then, Scott has been working with some clients on integrating DevOps. We’ve talked a bit about the challenges they’ve faced and we decided to do another webinar, going over some of the challenges clients face in moving to a DevOps style of software development.
Join me on Wed, Jan 18. Be sure you register for the event and set a reminder.
Today’s coping tip is to share a happy memory or inspiring thought with a loved one.
This is a memory for me:
This came up in a Google Photo memory and I showed my wife. Dec 31, 2020, after nearly a long year of the pandemic. Lots of isolation and separation from family and friends, but my wife and I took a few days together in the Colorado mountains.
We hiked, walked, skied a little, but mostly enjoyed being in a different place. The same circumstances (no eating in restaurants, limited contact with people, one of us in a store at a time), but still a break and a welcome one.
A happy time during a hard time.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I wrote last week about my travel, 23 trips in 2022. However, I’ve been gathering some other stats about my life and what I do, so I wanted to share a few of them here.
MusicI mostly listen to music on Spotify these days. One thing I loved are the Spotify wrapped updates at the end of the year. For 2022, I had these stats:
- 30135 minutes
- 1440 artists
- 68 genres
- 2946 songs
- Top Songs: Slide by H.E.R. with Pop Smoke, Hip Hop by Mos Def, Ms. Fat Booty by Mos Def, Hard Place by H.E.R., Into You by Fabulous
- Top Artists: Kanye, John Mayer, Jay-Z, Tupac Shakur, Kenny Chesney
Writing and SpeakingIt’s my job. A few stats:
- Editorials written – 151
- Coping Tips – 257
- Speaking sessions delivered: 34 talks
- Events: 19 events
- Blog views – 74,000+
- Visitors: 55k+
WorkoutsTaking care of my health is important to me. I’ve always exercised regularly, and at one point I ran every day of the year for a few years. I’ve relaxed a bit, and settled into more of a routine. I miss a few days and I don’t stress about it, but I make my best effort to work by body a bit most days.
Stats from 2022. I might have missed some, but these are the gross totals. This adds up to more than 365 since I do two things some days. A lot of weight days have cycline or walking or something as a warm-up
- Yoga: 89
- Cycling (indoor and outdoor): 84
- Weights 63
- Walking 22
- Elliptical: 22
- Swimming: 14
- Cardio classes: 2
- Rowing: 2
Total workout days: 232
The total days is low for me, 232, which is 63% of the year. My aim is 75% of the days, but I had ankle surgery this year, which threw me off for a month, and I spent way too much time on the road with travel and missed working out some of those days.
DrivingI don’t know exactly how much I drove, but the Tesla give me some stats and I’ve been tracking a few. For the Tesla, show about 17k miles this year, which is mostly me. For all the times my wife might drive (or the kids), likely I put miles on the X5 or Suburban (or a few in the Ram 3500), so this is not a bad set of mileage for me.
I also got to drive in two countries and a few states. Rough stats:
- Miles driven 17,000
- Countries: 2 (US and Portugal)
- States: 5 (Colorado, New York, California, Florida, Nevada)
- Top driving locations from the Tesla: Lifetime Fitness (135), Safeway (98), and a local gas station (89). Yes, I have a diet soda addiction.
ReadingI spend a lot of time reading books. They are an escape for me from life, and a way to improve myself. In 2022, I finished
- 101 books
- 35000+ pages
- Avg. screen time on Kindle app for Dec: 10 hours/week
Today’s coping tip is to see how many people you can smile at today.
Easy one for me. I find this makes me happier and my day better when I smile at people.
Wave, too. Even in big cities as I travel the world, a smile, a wave, and often I get the same in return.
Try it.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Sort of. Apparently, Microsoft will kill off IE in Feb 2023 in Windows 10. The lifecycle page notes that IE will be permanently disabled with an Edge update. For earlier OSes, it isn’t quite clear what will happen. However, extended support ends this month, Jan 2023, for Windows 7 and 8.
Does anyone use Windows 8?
I have used IE many times in my career, but the last decade of so I’ve shuddered every time I need to run it on a server or remote machine. It’s ancient technology that feels cumbersome, much like trying to get something done on a Palm Pilot instead of a modern smartphone.
I know there are lots of websites and apps that use technology that works in IE, and Microsoft is adding an IE Mode to Edge for those cases. Microsoft has a “what you need to know” article you can read, but it seems more like an advertisement for Edge than an informative article.
I could care less about IE, and I mostly don’t think I’ve needed it in years, but I do encounter that technology in a few places. Outlook is the main one where some sort of IE-based control is still being loaded for some authentication mechanisms. I think removing IE is one thing, but getting rid of embedded controls based on IE is going to be a much harder and longer-term issue.
I know so many developers that used various versions of the browser control in their apps, and I suspect there is no shortage of places that IE tech will continue to be a problem. I’m sure it works most of the time, but probably not always.
Hopefully, most of you out there use a modern browser for your work, and if you need one inside an application, you use something besides an embedded IE-era control. If you need an alternative, there’s an older Stack answer that might help. Please, use anything by an IE control. It’s time we let that technology retire gracefully in a museum somewhere.
Steve Jones
This weekend is Christmas, and likely many of you are not working hard today and will enjoy a long weekend until Tuesday. There are many other holidays at this time of year as well, and I hope you are enjoying the season with loved ones. I want to take a moment to wish everyone a Merry Christmas and Happy Holiday season.
As we come to the end of this year, it’s a good time to take stock of life. Work slows down for many of us, and it’s a time we think about family and friends. My wish is that you find yourself in a better position than last year. You are hopefully finding ways to cope and enjoy this world.
If not, then take stock of what isn’t working well and make a plan to change things. Life is short, sometimes much shorter than we realize, so move towards something that fulfills you, brings you happiness, and lets you enjoy the most of each day.
Enjoy the weekend and I’ll see you next week.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I do tend to travel a good amount as my kids have gotten older. The pandemic slowed things for a year, but only then. Someone remarked on this year being a lot of trips, and it has been. Not the most ever, but the most fun ever.
I decided to look at the travel numbers for the last few years, and see how this compares. Here are my trip numbers, for both business and personal, as best as I can tell.
- 2013- 18
- 2014 -17
- 2015 – 17
- 2016 – 16
- 2017 – 15
- 2018 – 20
- 2019 – 23
- 2020 – 1 (Las Vegas for my wife’s birthday, as the pandemic was starting)
- 2021 – 11
- 2022 – 23
Looking at this, 2022 ties the biggest year, but it didn’t feel as bad as 2013-2105.I don’t have exact numbers, but I know I was closer to 30 trips those years and felt more burned out.
I think because we have some fantastic vacations (Bruge, Hawaii, Venice, Lisbon) plus 3 volleyball trips and 3 trips to see my daughter in college.
Hopefully 2023 will turn out to be just as big and exciting with more places that we’ll visit and enjoy.
Today’s coping tip is to be generous. Feed someone with food, love, or kindness today
It’s often a family day today, as work ends and we prepare for Christmas. However, this year we invited over our volleyball team for a team building activity. We’re baking Christmas cookies and having some fun.
I’m helping out, and I set out some snacks and appetizers while the cookies get assembled along with some small gifts for the kids.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to buy an extra item and donate to a local food bank.
The food banks in my area are only open limited days, especially post pandemic. However, they often have volunteers at various stores this time of year. I’ve made it a point to get their list (usually with a photo to save paper) and go shopping.
This week I grabbed a bunch of extra items, aiming for the items they have less of, and adding in some fun things like extra cake/cookie mix and frosting.
Most everyone enjoys a treat this time of year.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I expected this to be an article from Glenn Berry. He often tries to convince me to upgrade. I think so he can justify buying his own new hardware.
In this case, however, this is a note from Tom’s Hardware that it’s a great time to upgrade your CPU. The article is a few weeks old, referencing Cyber Monday deals, and it notes that CPU sales are at a 30-year low. This means there’s a lot of supply and retailers are trying to clear out stock.
I don’t know how many of you find your CPU slow, or if you have access to change the CPU (often you can’t in a laptop). If you have a desktop, then you might not have a motherboard that supports the latest CPUs. Glenn would say upgrade your motherboard, but as I’ve seen in the past, this sometimes means memory upgrades as well.
Many of us work with server machines, but we aren’t responsible for the hardware. We often can’t even request different hardware. Glenn has written about which CPUs work for SQL Server, though not for a few years. Maybe this piece will get him to update that article.
SQL Server 2022 is the latest version, and the price as increased. The performance per core is more important than ever, so choosing the best CPU could have a big impact on the ROI of your database server. Even if you use the cloud, often there are choices in hardware, and expressing your reasoning for one over the other to the groups responsible for infrastructure can help ensure your organization gets the most performance for their billing.
And if you’re the person specifying hardware, you definitely ought to understand the differences between the CPU choices and make the best choice for your budget.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to listen wholeheartedly to someone else, without judging them.
I get the chance to talk with lots of people. I make it a point when I go to events or travel to get time with friends, or make time for people that come up to me and introduce themselves, and give them some undivided attention.
Sometimes these conversations go sideways, but most of the time people are very interesting in how they view the world. This tip made me think of the PASS Data Community Summit and an individual I met there. I was glad that I stopped and spent 10 minutes chatting with them.
After a session, someone came up to me and wanted to talk about DevOps and culture. This was a person who had studied psychology and culture in organizations and was now working with DevOps. They wanted to explain to me how they viewed the process, and where I was right, and where I was deficient.
I didn’t always agree with their conclusions, but I didn’t judge them. Their view and experiences were not mine, and perhaps they were right. In any case, I was happy to listen to their view and help broaden my perspective on something that is a regular part of my job.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to do something helpful for a friend or family member.
Easy one for me. My daughter came home from University yesterday, and I took time to go out and pick her up from the airport. And I brought a treat
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Ever since I heard about the SQLOS and all the work that went into producing an operating system for SQL Server inside of the host OS, I’ve thought that perhaps there might be value in specializing the environment for a database platform. Given the importance of databases, especially relational ones, and the need to get every bit of performance out of hardware, I wondered if we wouldn’t see a version of SQL Server that is installed directly on the hardware, without a host OS.
In some sense, I guess that’s what a PaaS database is in Azure, but I thought we might see that for the download-and-install version. I doubt that will ever happen now with the growth of the cloud.
Instead, maybe there’s a better way to improve performance without requiring SQL Server (or Oracle, PostgreSQL, etc.) to implement some of the OS features they’d need. Perhaps we could change the hardware around and use a SQL Processing Unit (SPU). The SPU is another specialized chip. Like a GPU for graphics, or even some of the other chips that are made specifically for mobiles, storage, or networking, this one would be focused on database needs. Apparently, there are a few companies that are researching how they might build chips that focus specifically on the types of computations that data analysis requires.
I don’t know how practical this is, or whether we’d see any major database vendor attempt to port to a new chip. Perhaps they’d add instructions that could use one of these chips inside a regular system, similar to how a GPU can offload work. That might seriously improve performance, something that a lot of our customers would appreciate. Especially if this happens without spending a lot of developer time rewriting old code.
That’s not to say that developers shouldn’t learn to write better code. They ought to, mostly so that their initial attempt at producing reports or batches of data handles a wide variety of data values and workloads without stressing the hardware. More efficient code is always the best way to attack any problem. Even if you had a very efficient SPU, chances are that your workload will still grow to overwhelm the hardware at some point, especially as the number of users grows.
Monitoring helps find problem queries, but it’s up to developers to change their habits, grow their knowledge, and produce better code sooner. Then we can use hardware to deal with the large numbers of users that need to access the system.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to give kind comments to as many people as possible today.
Not an easy tip for me to follow as I’ve been a little home bound after a lot of travel. I did get to the gym and the grocery, and I did make an effort to find something to say about people I interacted with that was pleasant and complementary.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Recently a customer was trying to sync up production and development. They’d somewhat lost control of both environments and wanted to build a plan of how to sync them. This post covers a process I suggested to them to tackle this challenge.
The ScenarioProduction is running. It’s got all sorts of objects, and most of them are probably valid. You’d be surprised how often I find broken code in production. Anyway, way want to ensure developers have visibility into what production looks like.
However.
Development is also in progress. They have lots of changes they’ve made in development, some of which need to be pushed to prod, some of which are in flight. They don’t want to just sync prod->dev or dev->prod.
How they get things deployed and keep track of work isn’t something I asked. I’m not judging; I know most people are just trying to get through the day.
So they wanted to know what’s different and mark those objects for someone to work on.
The SolutionThis is just one way to solve the problem. My approach here is to create a picklist of work that can be assigned to others. Since the customer has SQL Compare and is familiar, I used this tool to help me.
First, we back up development.
Next, I created a project that points from prod->dev. Worse case, we break development and restore it. We certainly don’t want to “break” production with too many mouse clicks.
For the sake of this demonstration, let’s say these are the databases:
- Production: way0utwest_prod
- Development: way0utwest
I set this up in SQL Compare, ensuring I have things pointing the right direction.
When I compare the databases, I see changes that are in all states. Some only in one or the other, some different, some the same. At this point, I don’t really care about counts or what’s different, so I don’t need to expand this list.
Now I select all objects and click “Deploy” at the top, which takes me to the next screen. Here I’ll create the script. Click Next.
At the next screen, I see the deployment script, but I’ll click the Summary tab. This gives me a list of what changes would be made to dev. Keep in mind we are trying to clean up development.
The summary gives me a list of the change at a high level. What operations on which objects. You can see I have a number of objects in the image above. In the upper right side of the image there is also a “Copy” button. Click this.
Now paste this into any editor. I’ll use Notepad. Now I have a list of changes needed for each object.
This is the picklist of work. Someone needs to go through this. You could paste this into Teams/Slack or anywhere, but really, this is the gross list of things to go through.
From here, they can take each item and create a work ticket for this. A work item in Azure DevOps Boards, a ticket in Jira, an entry on a Kanban board, it doesn’t matter. Use your work system to create these work items and then assign to people.
If necessary, repeat this process as you move forward until you have a list of things that ensure all objects from prod are in development. You also then delete things in dev that you will not deploy, and leave those items that are in flight.
You could also select the various categories in SQL Compare (different in both, only in one) above and generate separate pick lists for the types of work needed.
Ultimately the developers will know how to resolve these issues, so let them do the work. Just organize it for them.
A large part of the success I’ve had in my career has come from growing my skills, both technical and soft, throughout the years. I’ve always been driven to learn more and improve my ability to accomplish the tasks I’ve been assigned. Or those that I’ve sought out and tackled. A little initiative has been valuable in many successful reviews in the past.
There is a shortage of skilled IT workers. There are numerous job openings that I see in many companies, at least outside of some of the large tech firms. Many of those have been laying off workers, especially in some of their divisions that haven’t been as successful as expected during the last few years.
At the same time, a lot of companies are becoming more discriminating about who they hire. They don’t just need bodies, they need skilled bodies that can do the work they are struggling with, often in the DevOps, cloud, and data analysis areas.
While a lot of you reading this newsletter will have data skills, are they the ones companies need? Do you understand Git, DevOps, builds, and pipelines? Are you familiar with cloud technologies and gluing systems together remotely? Can you handle python, Spark, Power BI, or business requests to do something with AI/ML?
I’m often not surprised that many senior data people don’t have exposure to these technologies and aren’t sure how to gain skills. I am surprised that more employers aren’t upskilling existing people or reskilling them in new areas. It seems far too many companies don’t invest in their people, even in today’s world where hiring is both very expensive and time-consuming. It’s also frustrating when you can’t find qualified candidates.
There’s a good article on upskilling and reskilling and the importance of having a plan inside your organization. This goes hand in hand with a number of presentations from companies at the various DOES conferences over the years. The successful companies that embrace a digital transformation (whatever that means), DevOps, and the Cloud have ways to bring their current employees along. They invest in their people, requiring them to grow, but assisting them on the journey.
Not everyone wants to move to the cloud or DevOps or AI/ML. That’s fine, but my view is you have to embrace some change in your career if you want to create new opportunities for yourself and have choices in your future roles.
If your company doesn’t do enough to help you grow into new roles with new skills, maybe send them this article. Make a business case that you can do more with some help, and you’re willing to invest some effort yourself to grow. Managers know hiring is hard, and if you’ve been a good employee, I think they often are willing to invest in you.
If they’re not, then many you ought to think about that a bit and find out what you can do better so that they will invest in you. Not everyone makes it through corporate transformations, but many do if they are flexible and put in some effort.
Steve Jones
I caught a quick article recently that said you could have some fun in a Tesla with Santa Mode. You enable this in a few ways, but the quick one is to activate voice control and say:
Ho ho ho
This will change the UI. From the normal car to this:
Your car becomes a sleigh with Santa and reindeer and other cars are reindeer around you. The car also plays a song.
You can also activate a different song with “ho ho ho not funny”.
I made a quick video to show how this works as well: Santa Mode
A fun holiday trick for passengers.
It’s the holiday season and that means lots of promotions for various items to give as gifts. Black Friday took place recently and I had no shortage of advertisements that I saw for all sorts of things. From computers to televisions to headphones to any sort of tech gadget you can think of. Plenty of other types of gifts as well.
One item that kept cropping up for me was SSD drives. I saw an article in Tom’s Hardware about prices crashing to all time lows. That dovetailed with my part of the Data Community Summit keynote where I remembered Jim Gray saying 1TB would soon be US$10,000 in 1999 at the first PASS conference. I looked back at a few orders I’d made and saw I paid $150 for 512GB in 2017 and $125 for 1TB in 2020. Now I could get quality portable, SATA, or even NVMe drives for under $100/1TB. Incredible.
I thought about this recently as I packed for a trip. I used to carry 2 1TB drives in my bag, along with a couple of thumb drives. At a recent event, I realized I only had 1 thumb drive in my bag, which I hadn’t used for years. Packing for the next trip, I realized that my portable drives had been pulled out sometime during the pandemic and never returned. Despite almost 10 work trips in 2021 and almost 20 in 2022, I haven’t needed a plug in a drive.
In fact, I’m not sure the last time I used a drive with a wire. I’m used to getting everything through a network, even if it’s not a fast one. I wonder how many of you just live only on networks and never worry about using physical storage to transfer data. It seems that the idea of not really needing to use some physical medium to transfer information is becoming the norm rather than the exception.
I know there are still uses for physical drives. The Azure Data Box and AWS Snowball are used when large transfers will overwhelm a network. There are likely still some people who flip tape drives or mount and dismount disks as more storage is needed, but that seems to be a specialist role rather than something that many of us worry about. Especially with the cloud, it’s more likely that many of us may never need to touch physical storage again in our lives.
I don’t know if this is a good thing for the world, but I do think it is convenient. Knowing there is almost always a network around, and that we can make transfers between devices with wi-fi or Bluetooth without needing a physical cable is somewhat amazing. It’s a far cry from using multiple floppy disks or CDs to move data around. In fact, I had to search around for an optical drive as I realized I still have backups of pictures on DVD and none of the last few machines I’ve owned have a DVD drive.
The changing nature of storage still amazes me, someone that first dealt with tape storage as the medium for saving work. The world has come a long way, and I’m looking forward to what comes next.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to send a gift to someone new.
I have been sending gifts to my daughter at college every month or so, but I decided I ought to do this for others. I am sending out two packages. One to a family member, who I’m not naming here as they might not get it before this publishes.
Second, I picked a friend that has had some ups and downs. I’m both cheering them on and sending empathy, but I wanted to reach out and send a small care package to them as well.
Just a random gift. My family has moved away from everyone buying for everyone else, so I’m expanding that by picking a few people randomly to send things to throughout the year.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to support a charity or cause you care about.
My big causes are food, housing, and education for those that are struggling. I like to support these causes where I can. I had hoped to give time this year to Habitat to support their mission of shelter, health and time conflicted with my goal. I’m disappointed in myself here, and let life get away from me.
However, I can donate money, so I’m giving to them and the local food bank to help support their causes.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
The decision to upgrade database servers can be a complicated one that involves features, costs, and support requirements. While I think many people would love to run database systems for ten years, often there are concerns about support, which effectively ends after five years for SQL Server. While you can purchase extended support, is that worth the cost? A hard question to answer, but one I’ve had to confront lately. When do you upgrade a database instance?
I’ve discussed this with a few customers, but it also came up in the context of SQL Server Central. We run a SQL Server 2016 database on Windows Server 2016. Someone recently sent a note that since 2016 is out of support, they’d like to upgrade the systems. They were thinking SQL Server 2019 on Windows Server 2022, but with the release of SQL Server 2022 that doesn’t make sense.
I want to defer this process as long as possible. To me that means always aiming for the latest and greatest version. SQL Server releases roughly every 2-3 years, so this is the best time to upgrade for us. If we can upgrade before 12 months, we get 4+ years before we revisit this topic. If we were to upgrade to SQL Server 2019, then we’re already down to 2 years of support before we need to consider the topic again.
I think many DBAs would feel the same way, looking to test and certify SQL Server 2022 for their internal apps. This is one reason why getting the RC0 and RC1 releases of new versions for some initial testing make sense. The sooner you can upgrade in version’s lifecycle, the longer before you do it again. Of course, if you have more than 50 servers, you might just be upgrading every year anyway, as some system is always falling out of support.
Upgrades take time. There’s the time considering the decision, the testing a new system, the actual upgrade time, whether in place or migration, all of which eat up labor and time. Doing this for more than a few servers can become a full time job in some cases. That makes me start to really see the wisdom of using a PaaS service that’s evergreen.
For organizations where support matters, then upgrades are a fact of life and a regular occurrence. However, if formal support isn’t an issue, you might feel differently. The more mainstream you keep your feature usage, the more likely that you can go far past 5, or even 10 years, with your database system. I know there are still companies running 2008, 2005, and even a few 2000 servers. I don’t know that the SQL Server Central code would run on SQL Server 2000, but it might. Project Nami is using fairly generic SQL. If we were more concerned about database licensing, likely we’d have not upgraded the site from ASP.NET to WordPress.
For now, I think we’ll likely upgrade sometime in early 2023. I am not worried if things are delayed, but I do know that we also want to upgrade both WordPress and PHP. There is an order to getting all these steps completed with OS and database upgrades. Because of this, I want to ensure that we proceed soon and get things moving while my schedule is fairly light so that I can test and deal with any issues that arise. Hopefully there won’t be any, but I’ve learned to hope for the best and prepare for the worst, especially with software upgrades.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to offer to help someone who is facing difficulties now.
I’ve been very lucky in life. I find that success often has me associating with people who also have lots of success. Not always, but often.
Recently I had someone reach out to me, asking if I remembered another person from the past. I did and found out this person was getting laid off and was looking for a job. I told my friend to have them reach out to me, with permission to pass along my contact stuff.
I then offered to do some introductions with others, and see if there were potential opportunities. I sent a few messages, and I’m hoping one of them works out.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
It’s that time of the month, and I’m late. I’ve been on holiday for a week, so this is a quick post for T-SQL Tuesday. This month is hosted by Garry Bargsley, and is a fitting topic for the last month of the year.
Garry asks about end of the year data activities. Most of his examples are administrative, but I’m sure there might be some dev activities as well. I’ve got a few thoughts on each, some of which I’ve done, some of which I wish I’d have done in the past, but recommend now.
T-SQL Tuesday is a great place to participate in the community and a great way to show your knowledge and skills off as well for potential employees. Write your own posts on a blog, or somewhere like LinkedIn, Medium, or another site. If you want to host, ping me @way0utwest or sjones at sqlservercentral dot com.
End of Year AdministrationWhen I’ve been in the Operations side of data, there aren’t a lot of things I do at the end of the year, but I do find the downtime useful for some maintenance and cleanup. Usually we are doing well reactive stuff as business slows down, so I spend time on things that I’ve wanted to do all year, but haven’t had time to tackle.
The things I try to do:
- Index cleanup – look for dups, unused, etc.
- archive/delete data or tables – I try to clean data where possible. Less always makes things faster
- Chronic issues – Think about how I can solve something and prevent future problems
- Space planning – look over trends and be sure we’re ready for next year.
- Security – remove old accounts (or disable) as much as possible.
End of Year DevelopmentDevelopment tasks tend to be more tightly specified, and there often isn’t a good end of year list. However, similar to administration, one thing I have tried to do when times are slow is tackle things I’d want to change, but never have time. End of year is like this, as are some times after deployments.
Apart from code I might want to refactor or change, the big things I might look at during the end of year are:
- Branch cleanup – easy to have some of these hanging around.
- Pipelines – Use time to improve these, or ensure that they are noisy. Either change something, reduce tests, or try to avoid any unnecessary things causing the pipeline to go red.
- Learning – covered below.
Bonus – End of Year CareerThis isn’t specifically for SQL Server, but it could be. This is a good time of year to stop and try to assess how things went. It’s also good to look forward, and use slower times to make the future better. A few recommendations:
- Always have a list of things to read/learn/practice. Use slow times to work on something. If nothing else, tackle the Advent of Code.
- Assess your career – is this the place for you? The job, the employer, the field? Think about what things excite you and what don’t, what make days drag, or go by quickly.
- Plan for the future – Make a few career goals. I’ve done this for a few years, and it helps me continue to learn and grow.
Today’s coping tip is to contact someone you can’t be with to see how they are.
Maybe the one thing the pandemic did for me is make me think about other people more. More people and more often.
I’ve been randomly contacting people for a few years. Often people I’d expect to see at some point during the year, but knowing I wouldn’t. It’s been good to chat and share a few messages.
A couple weeks ago I found some old pictures from the past of various people. I sent them over in a message to say hi, remember a time we were together and check in on how they were doing. It was a fun few minutes out of my day.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Time is the most valuable asset I have most days. Time is often a limiting factor that dictates my priorities at work. It also often determines my stress and enjoyment levels in life. Using time effectively can lower the former and increase the latter.
A few years ago I read Making Work Visible, which talks about work flow and how time can be wasted in teams. The book covers the 5 types of time thieves, and it helped me to better think about how I organize both my tasks at work and in my personal life. I think that has sometimes helped me find a way to get more done with less effort, and less stress.
There’s an article from the author that summarizes the time thieves, talking about how to avoid losing time in a busy world. For me, I find that the WIP thief and the Unplanned Work are often the biggest thieves in my life. Usually when I haven’t done a good job of prioritizing and pushing back on low-priority items. Often, however, the person I’m pushing back on is me, so I’ve got to learn how to better manage myself.
Often the way to avoid many of these issues at work is with better visibility and communication. These are two things I’ve been thinking about quite a bit after the recent PASS Data Community Summit. While I didn’t have much to do with the event, I did speak, and I took on some extra speaking duties when others were overloaded. I could certainly see some time thieves at work in the run-up to the event during the last few weeks. Things to try and fix before next year.
We all need good technical skills, often strong ones when our organizations are trying to build, manage, and operate complex environments. However, there are other skills that can be just as important. Working in a team, coordinating work, being clear, and being transparent are all skills that can help us succeed without burnout. They are also key to finding these five time thieves and ensuring we get more done without requiring more resources.
I still don’t quite know how to manage my work and personal lives together, but I am working on getting better at working in a group, which is something I find myself doing more and more.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Today’s coping tip is to look at life through someone else’s eyes and see their perspective.
I have my own views, beliefs, and thoughts about the world. However, I often do try and empathize and view things from others’ perspectives. I think that has been one of my strengths over the years, in that I don’t just consider my view.
As an example, since it’s on my mind. While working with a customer recently, we were discussing the way in which they use our Flyway Desktop product. For them, then use nomenclature and a workflow that isn’t always what our developers think about. They also have a chaotic environment where work isn’t always completed in order. Or there are emergencies that need to be addressed.
I’ve been in those situations, so I stop and imagine past situations, pressures from management, and more. I’ve used that view when I go back to our product developers and discuss the ways in which we want to ensure flexibility for our customers, and allow them to deal with complex situations.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I’ve republished an editorial today from another author, called What do you do to relax after work?
I’m thinking about that a few weeks ahead, because I needed to schedule things. I’m on vacation now, hopefully enjoying Lisbon, Portugal with my wife. However, I thought this was a great question, so I decided to write something.
First, I’m not great at relaxing. I enjoy things, but I like to keep busy. Outside of work, I try to do some traveling with my wife when we can arrange things. Before the pandemic, we wanted to visit a new country together each year. We did that for a few years, before getting stuck at home. This year we visited Germany, Italy, and now Portugal together. We’d each been to the former two separately, but not together. This trip is new for both of us.
I coach volleyball with my wife as well. Really a second job, and a busy one for about half the year, but it’s a lot of fun for me and I do find it relaxing once in awhile, but I’m a type-A, charged person, so being busy and working on the problem of managing a team is fun.
I go to the gym a lot, I’m often fixing things around the ranch, and I practice guitar. Those can be relaxing, but not always.
Maybe the most relaxing thing is to read. I usually get close to 100 books a year, often sneaking in a few minutes here and there are I go through life and I have downtime. With the Kindle app, I always have books with me and I enjoy escaping life with a great book.
Today’s coping tip is to find out something new about someone you care about.
I took the time to do this at the recent PASS Summit. During events like this, I try to make time for new friends and also chat with longtime ones. I had the chance during one break to sit down with someone I’ve known for a long time. I don’t see them often, but I look forward to our meetings.
For the first time I got to learn a bit more about their family and life outside of work, as we’ve often just chatted about work. It was a nice break from a busy week and a fun time to spend time with someone I care about.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to be curious. Learn about a new topic or an inspiring idea.
My daughter pointed a new podcast out to me: What’s Her Name?
This looks at women in history who are often forgotten. As I age, and learn more, I know that the old adage is true: the victors write the history. A lot of men dominated history and they have sometimes ignored women’s contributions.
I grabbed a few episodes for my travels, to listen to as I’m moving around Europe this week.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to stop for a minute when you walk outside and just enjoy nature.
It’s getting cold in Colorado, but we had a day recently that was 50F/10C. No wind, and a pleasant day, so when I came back from the gym, I decided to sit outside for a few minutes. The dogs enjoyed me throwing a ball, but I took a few minutes to sit and look at the world. Enjoy the bright, big Colorado sky, smell the fresh air, and appreciate the silence of country living.
Quite a contrast from the last week in busy Cambridge.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s coping tip is to find a new way to tell someone you appreciate them.
My daughter is away at school, but she has a prescription that is still filled in Colorado. She hasn’t wanted to move it, so a couple times each semester I pick it up and mail it to her. I usually include some chocolate or a toy for her car inside, making a care package of the mailing.
I told someone this and they remarked they might do that for their parents, who they don’t see often. I thought that was a great idea, and a great way to show others you are thinking of them and appreciate them in your life.
So I put together a few care packages for others I know. They don’t need them, and there’s nothing really wonderful in there, but a few small gifts to let them know I care about them.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Out of town visiting a customer today, and it’s been a busy week. Plus, trying to get off for vacation, so you get: Does Speed Compromise Quality?
Today’s coping tip is to look for new reasons to be hopeful, even in tough times.
My life is pretty amazing most of the time. There are some stressful moments and annoying things, but overall, I can’t really complain about anything.
That being said, I have a lot of empathy for others, and I know that the inflation is a problem for many. I’m financially secure and don’t worry, but I also realize that I am lucky. Not everyone is in the same situation.
From my experience and travel, I’m glad to see some things starting to improve in the world. Fuel prices are lowering a bit, except for diesel. It’s not a perfect situation, but it’s better than nothing. I find there are more goods available in stores, which is always helpful for people that are just in time shoppers.
I’m also seeing lots of people posting well wishes, offering help, and doing favors for others. Community is important and it’s good to have people looking out for each other. In a very negative-world-view media storm, it’s nice to see plenty of people approaching life in a positive manner.
I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
I set goals at the beginning of the year, and I’m tracking my progress in these updates during 2022. Another month of busy stuff. A long trip for the Data Community Summit and then gone the last few days of … Continue reading →
It’s been a couple of weeks since the PASS Data Community Summit 2022. I was lucky enough to attend the event, and I had the chance to experience things firsthand for the in-person part of the event. I also had … Continue reading →
Today’s coping tip is to enjoy new music today: play, sing, dance, or listen. I’m on the road this week, so it’s music for me. Not a lot of time to play, and I don’t sing or dance well (or … Continue reading →
Today’s coping tip is to discover your artistic side. Design a friendly greeting card. A fun one for me today. I saw this just after returning from the PASS Data Community Summit. Since I had some fun with pictures, I … Continue reading →
Watching the evolution of SQL Server and the Azure SQL Database (ASD) variant has been interesting across the last decade. For a long time, ASD felt crippled compared to the on-premises product. The last few years, however, it seems that … Continue reading →
Today’s coping tip is to learn a new skill from a friend or share one of yours with them. I like learning, so in this case, I took advantage of my daughter being home. I asked her to help me … Continue reading →
Today’s coping tip is to make a meal using a recipe or ingredient you’ve not tried before. It’s the holidays, and my daughter is home. She’s GF, so I’m trying to make: Gluten Free Pfeffernusse Cookies I started to add … Continue reading →
It’s the last trip for me today. I head to the UK for a few work things and then my wife is coming over to take a quick vacation to Portugal. A hard life, I know. In any case, I’m … Continue reading →
Years ago my son asked me to buy him a copy of The Unincorporated Man. It’s a science fiction book about the future, economics, and sentient AI systems. It’s the first part of a series of four, and I’ve enjoyed … Continue reading →
Today’s coping tip is to broaden your perspective: read a different source of media. I’m off to the UK today. I tend to read books a lot when I travel and not embrace too much media, but I’ll try something … Continue reading →
I’m off today, enjoying the Thanksgiving holiday with family, so you get Securing Your Instances again.
Today’s coping tip is to connect with someone from a different generation. This tip makes me feel old. When I see “different generation” I used to think of someone older. Now I think of someone younger. At a large gym … Continue reading →
Today’s coping tip is to try a new way to practice self-care and be kind to yourself. One of the things that I learned to do better during the pandemic was take care of myself. I learned how to better … Continue reading →
Artificial Intelligence (AI) systems continue to pervade many industries, usually where there is a lot of data and human developers struggle to build solutions that handle the complexities of the problem. Often the experts in these subject areas can’t fully … Continue reading →
Today’s coping tip is to build new ideas by thinking “Yes, and what if…”. I tend to look for the holes, problems, limitations, and downside of various proposals. I’ve always been good at finding potential issues and mitigating them. It’s … Continue reading →
Today’s coping tip is to do something playful outdoors. Easy. Skiing today. First day of the season. I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me … Continue reading →
I saw a customer asking about Exasol recently, which is an in-memory, columnar database. I know nothing about it, and it might work great, but we don’t support it. I didn’t think much of it, as I’m sure the customer … Continue reading →
Today’s coping tip is to revisit a coping tip from the past and see if it helped you.. I came up with this one after I tried to do this: leave positive messages for myself. These have been popping up … Continue reading →
Today’s coping tip is to be creative, cook, draw, write, paint, make, or inspire. While I do play some guitar, I’m not much for painting. Certainly I write a lot here, and my creativity is often with cooking. I did … Continue reading →
Today’s coping tip is to try out a new way of being physically active. My routine for physical activity is yoga 2-3 times a week, weight lifting a couple times a week, swim once every week or two, and add … Continue reading →
When I first started work as a software developer, I knew that getting an environment set up where I could compile a project might take a few hours or a few days. The complexities of how people built software projects, … Continue reading →
I’m part of a Redgate promotion at the PASS Data Community Summit. They ordered some Lego Steve’s, which will be available at the booth. You can post some photos with the little Lego Steve, or with me, under #LegoSteveAtSummit and … Continue reading →
Today’s coping tip is to change your normal routine today and notice how you feel. I am a person who likes a routine. While I travel a lot and engage in different things, I still like a routine of sorts, … Continue reading →
Today’s coping tip is to sign up to try a new activity, course, or community. I somewhat did this early in the fall by choosing to go through the Head Coach certification for volleyball. While I’ve been in and out … Continue reading →
Many of us have learned how to research and get new information online. I certainly think the written word, whether from formal articles at places like SQL Server Central or from an individual’s blog, is very important to many of … Continue reading →
Quite a few of the bugs we see in production systems come from data that isn’t handled well. Perhaps the developer never considered this data, or another bug lets data into a system that should never be recorded. These are … Continue reading →
Today’s coping tip is to get outside and observe the changes in nature around you . Nature is changing in Colorado as winter approaches. It’s been a dry year, and that means that plants have gotten crispy and dry with … Continue reading →
I’m traveling this morning. My second to last trip of the year, and another long one. Today through the 20th I’ll be in Portland and Seattle. First is Portland, where I’m seeing lots of friends and speaking at SQL Saturday … Continue reading →
When I was younger, it seemed that everyone I worked with in technology knew how to build a computer. Most knew how to work with a BIOS, were comfortable with command lines, and could assemble complex compiler directives into a … Continue reading →
Today’s coping tip is to make a list of new things you want to do before the end of the year. Hmm, definitely some things to do, but what’s new? I’m not great with new as I like routine and … Continue reading →
There are a few community events at the Summit this year, but fewer than in the past. I know we’re all rebooting our conference experiences, and that can make motivation difficult. However, many of you go there to see friends, … Continue reading →
At this year’s Data Community Summit 2022, I am a part of the day 2 keynote. I’m excited to deliver some thoughts and memories of the event. I’m mostly excited the Summit still exists, and I am looking forward to … Continue reading →
Today’s coping tip is to be kind to yourself today. Remember, progress takes time. I am usually good at this. I work on many things that take time, so I am always looking at things over time. However, sometimes I … Continue reading →
My journey might be somewhat unusual, but perhaps not. I started writing articles on the Internet at a few different places before I started SQL Server Central. There were other authors online, and one of them was also in Colorado. … Continue reading →
This is another memory of the PASS Summit, this one an idea from Grant Fritchey, who wanted support the Women in Technology (WIT) events. And have a few laughs If you have a memory, share it with #SeattleSummitMoments. Only a … Continue reading →
Today’s coping tip is to be kind to yourself. Remember progress takes time. I’ve spent most of this year trying to better manage my weight and become healthier. I started the year at around 243 pounds, which for a 6’ … Continue reading →
The Community edition of Flyway has some nice basic features, and it works well for many people. However, it requires you to do a lot of the heavy lifting of building and deploying scripts. There are some advantages of the … Continue reading →
It’s time to look back at the 155th blog party. I was the host this month, asking about Dynamic SQL. I got quite a few responses, which I’ve gone through and summarized below. If I’ve missed someone, please ping me. … Continue reading →
Today is Halloween, a holiday of costumes, candy, and scary movies. My family has enjoyed scary movies over the years, though not necessarily on this date. However, the theme has me thinking about the scary situations that you’ve encountered, or … Continue reading →
Today’s coping tip is to find a new perspective on a problem you face. I don’t face many big problems, but I do face lots of small ones on a regular basis. One thing I’m struggling with is how to … Continue reading →
Apologies for the late invitation. A minor snafu has me hosting again. This is the monthly blog party where someone hosts and you all write a response. I’d like to think this is one where lots of you have a … Continue reading →
Today’s coping tip is to remind yourself that you are enough just as you are. This is interesting, as I’m not satisfied with who I am today. I’m not enough. I need to drop some weight, I need to build … Continue reading →
I was reminded this week that I needed to get registered for the Data Community Summit 2022 since I’m speaking. I also needed a hotel, so I took some time this week and got my flights, hotel, and conference booked. … Continue reading →
There was a short but interesting post on the value of seniority. It’s written from the perspective of someone that gets a new co-worker, but the co-worker is from the future. In fact, it’s the future you. The post is … Continue reading →
Today’s coping tip is to choose to see your mistakes as steps to help you learn. I preach this with the kids I coach. Mistakes will happen, but let’s learn and grow from them and don’t dwell on them as … Continue reading →
Today’s coping tip is to free up time by canceling any unnecessary plans. I. Suck. At. This. I don’t like to blow things off, and I do like to keep busy. I also don’t schedule a lot of things, and … Continue reading →
When Microsoft started to talk about Intelligent Query Processing (IQP) before SQL Server 2017, I wasn’t sure what to think. There was a diagram with 20 things on it, and only 5 were highlighted (you can see that in the … Continue reading →
Today’s coping tip is to write down three things you appreciate about yourself. I don’t mind self-evaluation, but I struggle a bit to publicly talk about things I do well. Still, it’s a bit of coping for me to become … Continue reading →
I have a mechanical gaming keyboard, not because I’m a gamer, but I did want to tactile, mechanical feel and I like the idea of lights on the keys. I’ve enjoyed the experience, but the software leaves something to be … Continue reading →
Most of us are used to a database that lives on a server somewhere. It might be in our data center or a VM that exists somewhere, but it’s really an on-premises type of infrastructure. Even if the VM is … Continue reading →
Today’s coping tip is to find a new way to use one of your strengths or talents. I asked someone for strengths recently. Most of those items are things that I use regularly in the same way. However, I did … Continue reading →
Not part of my Tesla series, but this is related as I had an interesting electric car experience while traveling in London. This is part of a series that covers my experience with a Tesla Model Y. Booking Uber I … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Licensing is always a complex discussion with SQL Server. Depending on the way you run it, the edition, the version, whether you’ve purchased Software Assurance, and more, you might struggle to ensure you are in compliance with Microsoft’s terms. This … Continue reading →
I was honored to speak at Future Data Driven last year. This year has a great lineup with some fantastic sessions on data related topics. Register today for the Sep 28 event This is online, so you can attend from … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I’m still in Hawaii. Hopefully things are amazing. At this point, I’ll have been on Maui for 5 nights and starting my fifth day in paradise. Possibly I’ve posted pictures on Twitter or Insta, so you can follow along there. … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
This is the last workday for me this month. I’m off to Hawaii for a break, so only coping tips for the next week. Aloha.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. Here are some hints to get started. I was demoing something recently and needed to show someone how to … Continue reading →
I just paid off my mobile phone. In my case, this is a phone from Google Fi, which provides fantastic service for me. The phone works well in any country I travel to, without any extra roaming costs outside the … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Recently I was testing a feature in SQL Server on 2017 and 2019. There was supposed to be an improvement across versions, but I didn’t see it. Then I realized that I was on SQL Server 2019 CU 2 on … Continue reading →
This is part of a series that covers my experience with a Tesla Model Y. One of the reasons I have some data capture taking place on my own systems is in case some vendor I use has a DR … Continue reading →
We’ve run Kubernetes inside Redgate for some research projects (like Spawn) and we are building some skills running this orchestrator. At the same time, we’ve had no shortage of challenges in keeping the clusters up at times, patching, fixing issues, … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I wrote recently about making a HEAD utility to find the top few lines from a text file. I used Powershell and scripting to make this work from any Windows command line on my machine. Someone asked about TAIL, which … Continue reading →
When I was starting my career, I expected to be a programmer. That’s what people who wrote the software were called. At some point they adopted “developer” instead, shunning the programmer label. Now I see software engineers has replaced developer … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
If you attended my talk at SQL Saturday Los Angeles 2022, then you can get my slides here: Adopting a DevOps Process for your Database.
One of the things I did often in my first career job was create utilities that we could use as a network support team. I was an intern coming from university, where I used SunOS and Solaris all the time. … Continue reading →
Many of us know that reducing financial debt in our lives leads to a bit more security in our minds. We are better able to cope with unexpected expenses when we have the ability to get credit or pay for … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
This is part of a series that covers my experience with a Tesla Model Y. I subscribed to the Tesla FSD in July, since I had a couple trips planned where I’d be driving longer distances on the highways. I … Continue reading →
I was giving a talk on DevOps recently and one of the questions from a person in the audience was how to get others to buy in. This person also had complaints that “DevOps” wouldn’t work because their boss wasn’t … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Thanks to everyone that attended my sessions today at VS Live. I have my slides and code available for download at the VS Live site and here. The Serverless Azure Database – PPT zip Adding Graph Structures to Your Database … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I’ve been working to better understand graph databases and where they can be useful. There is a file from Neo4J that comes with the Desktop and contains a data export from Northwind. This looks like this when you open it: … Continue reading →
One of the sales managers at Redgate Software posted an origin story about how he came to work at Redgate Software. It’s an interesting story, and while I don’t work in sales or have the same origin story, I don’t … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
This is part of a series that covers my experience with a Tesla Model Y. I wrote recently about the costs for me to charge at a Supercharger. The USD$0.43 was higher than I expected, but lower than the USD$0.58 … Continue reading →
I like the feedback system that Microsoft built for SQL Server. This used to be the Connect system, but all the bug reports and feature requests are now at feedback.azure.com. That’s the place where you can send notes to Microsoft … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I’ve been tracking my usage and comparing that with the estimates for my solar power system. I wrote about the database design and tracking the usage and some of the estimates. In this post, I want to look at the … Continue reading →
One of the more successful uses of AI (artificial intelligence) has been in the medical field. It seems that there is a tremendous amount of data, high variability in some aspects of the target of the data (the patient), and … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I use Libsyn to store podcast files. They’ve been a reliable service for me for years. Once in awhile uploads are slow, but things seem to work. The other day I went to upload a file and got a status … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I work often with Azure DevOps. I have enjoyed the platform and it does what I need. I also work regularly with GitHub and GitHub Actions. Those rebuild SQL Saturday and SQL Memorial when I need to make changes. It … Continue reading →
The last two years have changed the way many of us work in technology. The pandemic allowed many of us to work remotely, a challenge that many companies are struggling with. Or perhaps, management is struggling with and employees appreciate. … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Microsoft wants more people to move their database workloads to Azure. They are constantly adding new features, capabilities, even tools to help people move databases to some part of Azure. It’s working well, as the latest quarterly report shows tremendous … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I was honored to attend the first SQL Saturday in New Jersey last weekend. I consulted with and helped the organizers get the event going and executed. I made a few notes and got some pictures, some of which I’ve … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
The SQL Solutions Group asked me to do a webinar with them, which is still an honor for me. They’ve been working with various clients on DevOps development practices, and they sometimes ask my thoughts on an issue. I’ve known … Continue reading →
I found this article to be an interesting look at how we might add ethics to AI systems in one area. As the article points out, “… today there is no broadly accepted AI ethics framework, or means to enforce … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I had a great talk today at SQL Saturday New Jersey 2022 on DevOps and databases. Good crowd and good questions. Got caught in a spotlight picture as well: The slides are available for download.
This is part of a series that covers my experience with a Tesla Model Y. I finally had a charge for a Tesla Supercharger. These re the public, high speed chargers that Tesla has put all over the world, which … Continue reading →
I wrote the other day about a culture of allowing mistakes. We know mistakes are going to happen, so we ought to accept them. Even stupid ones. I make them at times, my wife does, my kid does, so I … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Building a culture that promotes teamwork, efficiency, retention, and other positive attributes can be challenging. While many organizations want accountability and responsibility, it is easy for managers to tip over into a blame-and-chastise pattern. When someone makes a mistake and … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
SQL Saturday is coming back in-person to Los Angeles. I’ve been to this event a few times, and was sad when the pandemic pushed it virtual. However, 2022 is the year to restart. I’m speaking, and my flights are booked. … Continue reading →
I learned something new recently. I can search in SQL Monitor for a database name, not just a server or instance name. I tested this over at monitor.red-gate.com recently after a developer mentioned this feature. If you look at the … Continue reading →
I’ve had a number of jobs in my career. In many instances, I left on good terms, and I’d go back to the organization if there was a job that suited me. I’d like to think that many of these … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
SQL Monitor has improved a lot over the last couple of years. We have multiple teams building features and addressing issues, and each month when we have a readout of changes, I’m impressed. Since we update the produce every week … Continue reading →
When I watched Star Trek as a kid, I was amazed by the technology. Talking to the computer, the touch screens, the handheld communicators. We have most of those devices now, without the space travel. Hopefully that will start to … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
We added a solar system to the ranch as a way to hedge against future costs as well as require less energy from the utility. I don’t plan to, or think, that everyone can get away from using utility power, … Continue reading →
Data storage has always been a concern for data professionals. Early on in my career, we dealt with large ESDI, IDE, and SCSI drives, all of which would fail unexpectedly in servers. Sometimes after a few years, sometimes after a … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Someone asked me recently about my travels this year, and I listed out the trips I’ve taken. So far, I’ve been to these airports on trips away from home: LHR SYR LAS ORD RNO LAX JAX LHR BRU AMS AUS … Continue reading →
I guess that I was a remote employee that needed to onboard at a new company at one point. When Redgate Software bought SQL Server Central, I lived in Colorado and the company was (mostly) in Cambridge, UK. However, I … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Many of us work with data in some way that helps a customer better understand data, use it to make a decision, or support a some conclusion. The way we present data (or help others present data) can impact how … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
One of the new language features added in SQL Server 2022 is the GENERATE_SERIES function. This allows you to generate a SELECT * FROM GENERATE_SERIES(start=1, stop=7) This gives me a simple sequence of numbers in a result set, with the … Continue reading →
This is part of a series that covers my experience with a Tesla Model Y. When I look at my top destinations in the Tesla, I find that home is the top one (327) and my gym is second (102). … Continue reading →
At Techorama, I saw a keynote from Derek Martin of Microsoft. The talk was called Pain, Grief, Perseverance, and Technology, and it’s worth seeing if you can find it live or recorded. David talks about his growth in life from … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Thanks to everyone that came to my talk at VS Live today. I’ve uploaded the slides to the blog. Here are the two decks: Using CI To Prevent Database Problems The Serverless Azure Database Code for the CI Talk is … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. Here are some hints to get started. I had a customer recently that was looking to work with Data … Continue reading →
This is part of a series that covers my experience with a Tesla Model Y. I went to Europe for two weeks. My wife joined me a week in and left our Tesla at the airport. Once she got on … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Last week I attended DataGrillen 2022 in Lingen, Germany. This was my first time at this event, and I had a wonderful time. This despite the fact that I was a bit limited with mobility, as you can see below. … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I was talking with a friend recently about technology. This individual is a person focused on business intelligence, originally a developer, but now an architect and consultant. They have a fair number of clients and have worked with them to … Continue reading →
I was doing a little experimenting with graph databases. My goal was to run Neo4j in a docker container, but it wasn’t quite as simple as I expected. I started with a How-To, but it wasn’t enough. This post covers … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I set goals at the beginning of the year, and I’m tracking my progress in these updates during 2022. A few days late, and I missed April, but I’m trying to get back on track for May. It’s been a … Continue reading →
I’m still at DataGrillen today, enjoying German food, fun, and learning a little data stuff. You get The Nightmare Letter that I thought might be more of a problem. I don’t think this is as wide a problem as many … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Awhile back I wrote about my crazy schedule. I’m about halfway through. The last couple months have been: London (SQL Bits) Syracuse (daughter visit) Las Vegas (volleyball) Chicago (volleyball) Los Angeles (work) Reno (volleyball) Cambridge/Brussels (work/Techorama) ankle surgery Amsterdam/Lingen (Datagrillen) … Continue reading →
Back on the road this week, for DataGrillen this time. I’m in Germany and hopefully enjoying the trip as I travel with a boot. While I’m at the event, you get to read about Production Subsets, something I think is … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I’m off again. By the time this publishes, I should be in Amsterdam (hopefully). I’m traveling to DataGrillen, but because I had surgery and in a boot, I’m moving slowly. This week is traveling to DataGrillen and back, hopefully without … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. It’s Memorial Day in the US. A holiday, though I’m off on a trip to Germany today. I wanted … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
This was a strange week for me. Traveling in Brussels with my wife and prepping for a few presentations. Then Techorama, an even more 2019-era conference in Belgium where I spoke around a few customer calls sprinkled in during the … Continue reading →
I had a great time at Techorama. I’ll do a writeup on the event, but for now, here are my slides. The VCS Primer Adopting a DevOps Process for your Database There’s no code, but if you have questions, please … Continue reading →
Here are the slide decks for my sessions at SQL Saturday Jacksonville 2022 Continuous Integration Using Local Agents Adopting a DevOps Process for Your Database
I leave this morning for nearly two weeks. I head to Jacksonville for SQL Saturday Jacksonville 2022, then I’m off to the UK to visit Redgate’s offices for a week and then I spend part of the last week in … Continue reading →
Hacker News is a forum for experienced developers. While I think plenty of beginners lurk there, the discussions seem to be dominated by those who have worked in the technology or software industries for years. I saw an interesting article … Continue reading →
This is part of a series that covers my experience with a Tesla Model Y. I wrote recently about the Tesla and the charge loss while sitting at the airport. While a little worrisome, I knew it was something I … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. I wrote a SQL New Blogger post recently on running totals and used some images to show data. A … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips … Continue reading →
It’s that time of the month again for a T-SQL Tuesday blog party. This month is hosted by Kenneth Fisher, who has hosted a few times in the past. I like this topic, and it’s a good one for me … Continue reading →
I took the beta exam for Cosmos DB a few months back. I did this as part of the MVP program, as they were asking people to evaluate the exam and give feedback. I’ve lightly used Cosmos DB, but I also thought I might get some practice with the platform and taking an MS test, which I haven’t done in years.
I passed!
I got a note today that the results for DP-420 were in. This is the Designing and Implementing Cloud-Native Applications Using Microsoft Azure Cosmos DB exam, which is way out of my area.
To be fair, I didn’t want to just take this blind, so I spent about a week and a half studying regularly each day, going through the learning path and understanding the different aspects of CosmosDB. It was interesting and confusing, but I acquired a decent understanding of what the platform is and can do.
What I didn’t know were some specifics like how many RU/s might be available for a particularly sized contained. I did, however, learn about the different types of containers, accounts, hierarchies of things, etc.
Feedback I like the beta exams because you get more than the 5 minutes or so to give feedback, and can do so for many questions. While I think the exams do test some knowledge in most cases, I also think that they can mislead people at times. I also think that some questions are really fact memorization.
For example, if you asked me what is max size of a serverless container, that seems like a reasonable thing to know. If you’re planning an install, you should know that serverless limits things.
However, this is a memorization fact. It’s a documentation thing you can look up in seconds, and by the way, you should. That number could change any day as Cosmos DB evolves.
There were 2-3 questions like that on my exam and I gave that feedback. I don’t have a better question, but knowing that a container is 10GB or 50GB seems like a poor test of knowledge.
There were a few issues I had with wording, but overall the questions weren’t bad, IMHO. They were hard. Unlike many of the other SQL exams, I found most of these to be hard and was unsure of if I really knew the answer.
At SQL Bits a few weeks back there was a community keynote on Friday. Ben Weissman and Rob Sewell put this together as a fun way of involving others. They invited a number of people to each take 5 minutes and talk about their favorite part of the data platform, which encompasses a wide variety of technologies and products. There were a few people who chose Purview, Azure networking, and quite a few ways of using Power BI.
Me? I choose the Create button in Azure. Really, Azure itself because of the amazing array of things that you can deploy in minutes. It’s stunning to me to think back to the mid-2000s, when I first saw the Azure platform at TechEd and a Microsoft employee demo’d a key-value table, literally a two-column table. They were trying to sell this as a great way to deploy lookup data to mobile sales applications at scale. I was less than impressed.
This year I deployed a Synapse workspace in less than 10 minutes. Maybe less than 5. I know little about Synapse, and the idea of building a data warehouse and analytics platform, loading data, and running queries isn’t something I’d think I could do in 10 minutes, but I did it. I’ve found containers to be similarly useful in quickly getting something up and running, but even they require Docker or some other software installed and configured. In Azure, I can set things up in minutes.
In the last year I’ve set up numerable resources, from databases to web apps to a Kubernetes install. All in minutes, which is quicker than I could do something on my local system except for creating databases. If I had prepped things or had more knowledge that might not be the case, but the lack of needing to prep or learn a lot to get something set up is amazing in and of itself. This is especially true when I want to experiment with a technology or a new solution. I can quickly build an array of systems with much less effort than on-premises.
I don’t know how many of you use the cloud at work, but I find it amazing. It’s one part of the data platform from Microsoft that I really appreciate, especially as I’ve tried to work with MongoDB, Redis, and other technologies at home. The cloud makes things easier, even with databases.
Today, I’m wondering what your favorite part of the data platform might be. Is it something in SQL Server? Maybe a related technology? Perhaps something else that helps you work with data these days. Let us know what data platform thing is exciting to your career.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to tune into your feelings without judging or trying to change them.
This is an interesting idea. I saw this in an essay from David Perell, with the idea that we want to experience negative feelings (anger, sadness, grief), but let them wash over us and move on. I have been trying to do this.
On the day I saw this tip, I was feeling upset, sad, angry, and more about some of the community issues I’ve seen recently. Some of these were from empathy for others, some were because I needed to deal with things.
However, I also needed to both move forward with my day, and work on how to work with the issues. I’m torn how I feel about the issues, about people, and what I should do. All of those things are valid. I am not trying to change the feelings, but just deal with them.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to notice what is going well, even when things are difficult.
Overall my life is amazing. The problems I have are just minor annoyances, really, and I try to remember that. I’m not always successful, but my wife helped me remember that recently.
We took a trip to see my daughter recently. This was a long weekend, leaving Thur, returning Monday. A snow storm was scheduled for Monday, so my wife was concerned and wanted us to change and come back Sunday. My daughter has been having a hard time and didn’t come home for Spring Break, so I wanted to see her, support her, and get some time.
All of this was on my mind when we boarded a plane for a 945a departure at 910, and more on my mind when the 12 of us that got on packed up and left at 915. There was some issue with the toilet being broken, so we needed a new plane.
Not the end of the world, and as I fly a lot, usually a couple times a year there’s some issue. In this case, we had to change gates and wait for a plane. It was scheduled to depart now around 1110. The plane didn’t arrive at the gate until about 1045, and after unloading and cleaning, we didn’t load until around 1130. We moved away from the gate, went to de-icing, and things were looking up. Slightly stressed as I had a 2 hour drive on the other end and wasn’t sure we’d get dinner with my girl, but that’s a minor issue.
We got to the runway and then turned around and returned to the gate. Some sort of brake issue. I like brakes on planes, so I took it in stride. A little stressed, but OK. At the gate, they announced this might take some time, so they let people get off. I’d been upgraded (again, my life is easy and amazing), so my wife drank mimosas while I read a bit. I tried to relax, but it was hard.
They announced it was a sensor, so they brought people back on. Then this wasn’t the sensor, so more mechanics, and people off the plane again. We stayed on again, more mimosas for me wife, but stress for me. At this point I was wondering if we’d leave. If we pushed to Friday and left Sunday, that would start to feel silly.
My wife reminded me that we were getting time together, whatever happened. We could find a way to go out another week or have my daughter come home. We didn’t have any big commitments, so I should relax.
I tried, and it was a good reminder that things were going well. I was in a first class seat. I could have a drink (I declined, citing my diet), and I was with my wife on a trip. I didn’t relax much, but I did at least see the good things.
We did manage to finally go, six hours late, and we arrived at the AirBNB at 1130. Too late for dinner, but we had a great weekend with my daughter, and we stayed through Monday. Everything worked out.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
Lots of people have never worked with LocalDB, which is an in-process version of SQL Express. No service account, just a SQL Server instance running with your app. It’s a nice lightweight way to get SQL Server running quickly without a hassle.
This is a SQL Server Express version, but the bare bones for development. This post looks at how you can get this running.
This version of SQL is installed with SQL Express, and with Visual Studio. If you look in this path: C:\Program Files\Microsoft SQL Server\150\Tools\Binn, there is a SQLLocalDB.exe. You can see that here.
This is my SQL Server 2016 version of LocalDB. I can start a new instance by calling this with the CREATE option. I can give this a name as well, as I might want to stat multiple instance for different apps. Here I’ll create an instance called app1.
SQLLocalDB create app1
I then call the same command, but use START instead of CREATE. You can see this reports as started from the CLI. I also add the INFO call to get status.
Now I can connect. I use (LocalDB)\app1 to connect:
I see I’m connected to a version of LocalDB then:
Now it’s just an instance of SQL Server I can use.
SQL New Blogger I needed to check something for a customer and realized I hadn’t started LocalDB in a long time, so I needed to check the docs. I spent 10 minutes putting this post together.
An easy type of post for any of you out there. Learn something, try something, write something.
I love this idea from Ken Fisher: saving your work. I don’t act as a DBA anymore, but when I did, I did something similar. We often logged the scripts we used in a file, as a part of a log, so that if we broke something and another DBA got a new ticket, they could check what you had done. Over the years, we tried two different methods. First was using the desktop of the instance itself, since we often went to a room to log into the server in those days
The second way was in an Exchange public folder, where we added a new entry for each day. This way we could note the server and the scripts run. Since most tickets were dated, we could easily find the scripts if we were looking at a ticket. Since a user often updated or re-opened the ticket, we could use the public folder as a central note location from the DBA team. We could even point to this folder for our ISO and SOX auditors to show them what had been logged by people who supported the systems. Not a perfect auditing system, but one that often was accepted by auditors.
However, the one thing missing in there, from my perspective, is version control. While I think it is important to track these scripts in a team of DBAs, I also think we want to ensure that as we grow and change these scripts, we know how and why. Junior people can learn from changes made by senior ones, and if a DBA alters one of these scripts and breaks something, just as a developer might refactor code and introduce a bug or break functionality. After all, these scripts are code.
If there is a problem, we want to be able to roll back, which means that we ought to save these scripts into a repository of some sort. While I like the idea of a share that all DBAs can access, I more like the idea of a (secure) Git repository that can be downloaded anywhere, provides a second backup, and can be audited over time. All of these are important features that any enterprise should implement, especially one that is regulated. We want to protect ourselves if a DBA gets hit by the proverbial bus.
I like collaboration, sharing knowledge, and tracking the work you do in a team. It’s important for raising the skills of everyone on the team and helping new members get up to speed quickly. This facilities consistent results, and if done using a tool like version control, helps ensure that your scripts are backed up in a way that preserves the knowledge in your code through any changes made by the team.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to listen to a piece of music without doing anything else.
I saw a list of acoustic guitar songs recently and on it was Babe I’m Gonna Leave You by Led Zeppelin. I listened to a lot of their music as a youth, and I’d forgotten about this song. So, I pulled it up on Spotify, closed my eyes, and just listened.
Like many of their songs, there is some beautiful, peaceful guitar part that moves into a rock opera. An interesting lick that quickly gets buried inside of a heavy bass and drum part. At times, the loud singing is at odds with the guitar.
An interesting piece.
Another one, which I actually heard while swimming recently, was Starship by Kanye West. I was moving along and found myself distracted by this tune, actually hearing the words. I know I’d heard the song before, but I don’t know I had listened to the words, but enjoyed that one, completely distracted from the effort of the monotonous laps.
In the world of cybersecurity, a red team is a team of hackers that try to infiltrate a company, but at the request of the company. The idea is they look for vulnerabilities and issues and find them before criminal hackers do. They are the opposite of the blue team, who is trying to stop the red team and create defensive measures that prevent the red team from accessing data.
Many large companies use red teams. Microsoft maintains a red team (and a blue team) that are constantly competing to break into their systems and defend them (depending on which side you are thinking about). I’m sure Amazon, Google, and other large companies do the same thing. It’s an interesting idea, though I bet this is a lot of repetitive work where you constantly repeat similar attacks with slight variations. There certainly is some creativity and research as well, and some acting if you social engineer situations, but it’s not the type of work I would want to do. I doubt it’s as exciting as Hollywood movies portray hacking.
An organization could assemble a red team from external resources and use them to evaluate the security of your software, your infrastructure, or even your people. There’s an article this week on getting started with a red team. Since companies are seeing more and more attacks against their systems, I would expect more to be proactive and either assemble or hire someone to test them. In fact, I bet there will be lots of cybersecurity people setting up their own companies to help here.
Many of us might feel we are careful with security, and that we check for issues. I’m sure we do some of that, and many of us know how to secure things well. However, it’s easy to make a change and make a mistake. It’s easy to forget to include a group, or include the wrong group, in a security ACL. It’s easy to forget to check a setting or leave access open while we test and forget to go back and secure it properly.
Attackers think differently. We can learn to do this, but it might be helpful to have someone else doing the attacking and then giving us a report on what to fix.
Steve Jones
I read Robert Cain’s post on his use of technology to improve health, and wrote an editorial with a few thoughts. I’ve been doing some similar things for years, so I decided to add some notes on how I use technology and devices for looking at my health.
I’m not fanatical about this, and I’m not trying to capture everything in detail, and this might not be good enough for many of you, but this gives me some information with a minimal impact to my life.
Water Bottles for Hydration I’m going to start here because it’s the easiest thing that helped me. A few years ago I found myself feeling dehydrated on some days. This was especially true after traveling away from Denver. Many place aren’t as dry as home, and I find myself not drinking enough water when I return.
I found a simple way to help me remember was to just fill multiple water bottles and set them on my desk in the morning. Throughout the day I could see if I had 4, 3, 2, or 1 full bottle of water. This helped me to increase my intake at home. When my doctor told me to go from 64oz to 100oz, I changed 2 24oz bottles to 32oz ones.
On the road, I’ve started to carry a bottle and do what I can. I could use an app, and I might try Robert’s app for my next trip, but I don’t want to get annoyed. The habit at home has helped me to be more cognizant away from my desk.
If I go to the gym, I just grab one of the four and take it with me. This is a good low-tech approach for me.
Garmin Forerunner for Exercise Redgate gave me a 10 year anniversary gift of a Garmin Forerunner 645 smart watch. I’d been considering a few, and this was a good one for me. The main advantage I found with this watch is the battery can last 4-5 days if I don’t use the GPS. Since most of my exercise doesn’t use this, not needing to charge this daily is something I appreciate.
Many of the Garmin watches (and others), allow you to track exercise. I have loaded a number of exercise tracking profiles loaded. For me, I can click a few buttons and select from yoga, weight lifting, biking, swimming, etc. I start tracking when I start and end when I end. For a few items, I can click a button that marks a set, lap, etc.
For weight lifting, this tracks sets, but also tracks rest time. I try to take no more than a minute between sets, so this is handy. It also helps me remember if I’ve done 5 or 6 sets. That is nice since I might be looking at email, reading, etc. while I work out.
After exercise I glance at my results to see my average heart rate, my level of effort, etc. As a couple examples, here is what I see for swimming:
I tend to find 2:00-2:05 is pretty pace for me to stress myself. If I get above 2:06, either I’m slacking, or my body is worn out and I don’t realize it. A minute or two here, against what I thought my effort level was helps me evaluate my stress and sleep.
For weight lifting, the app guesses pretty well for some exercises, but not all of them. I could edit these, but I don’t bother. I’m more concerned with time and heart work here.
I usually only look at my effort level for yoga afterwards. Here you can see I wasn’t working hard enough on this day, but I can decide then how I view that. Sometimes I need a break, and that’s fine. Sometimes I see this and it reminds me to work harder tomorrow.
Heart Rate I don’t have a history of heard issues in my family, but I do think that heart rate can be an early predictor of other issues or increasing stress. When I got my watch, I started tracking my heart rate. It was interesting to see what my resting heart rate was, which was around 54bpm 4 years ago.
I usually look at the 30 day views, but I can see a longer term, which you see here. My 1 year average is 51bpm. I’m at 48bpm for the last month, which is an improvement. I think that’s been dietary changes.
Other Metrics The watch tracks sleep, which I sometimes glance at across a week, mostly just to see if I’m resting enough. I don’t need to this be amazingly accurate, but more it helps me think about myself if I’m feeling rundown.
I started adding my weight in here every day or two. I step on the scale, get a number, and then click “add” and “weight”. The app remembers my last weight, and if it is unchanged, I can just click OK.
The watch tracks steps, floors, stress, and probably more. I don’t bother. I sometimes look at steps if I haven’t done any exercise that day. I can then decide if I need to make time to exercise, or live with the amount of movement I’ve gotten that day.
Lose It I didn’t use Lose It, but my wife liked it. She tried Noom, and liked it, but felt it was a little too intrusive in her daily work. A friend recommended this, and she liked it. She could type into search the food, or scan a bar code.
This helped her track calories and make decisions on what to eat. I, however, find this to be too much data entry, and then too much work to then try to plan future meals for the day.
Health Records I am in the UC Health system in Colorado. My insurance covers this group of physicians, and there is a nice integrated system that captures appointments and notes, but also includes test results. They do graph and track things across time, but I’ve started to download and save results.
As I get older, knowing how your body is changing might be useful for future doctors. I want to have a better view of my health, so I’m managing this. I don’t have a great solution. I am lightly testing a few apps, but recommendations appreciated.
MapMyRun There are lots of places to track long term exercise. While Garmin does track this, I always worry about a data breach, failure, or some issue when I don’t control the data. I used to manually enter things into Mapmyrun, and that was where I tracked my running streak.
There is a link from Garmin to MapMyRun, and my watch now updates both places. I have over a decade of exercise in MapMyRun and I periodically download my data to have it.
Community I haven’t really participated in the community aspects of either Garmin Connect or MapMyRun. Neither appealed to me, but I do know some people use these to motivate each other, or they attack challenges that are available.
You might find these things useful.
Summary Health is important to me, and I hope to you. I don’t think that you need to obsess or make this the most important thing in your life, but I do think this is something to pay attention to, especially once you get to 40 or 50 years of age.
We don’t necessarily need to track everything, but we ought to be aware of some metrics and how they might change over time.
Passwords aren’t going anywhere. While I would have thought there would be more advances by this time, and there are, the basic password is still required in many places, especially for resetting an account. Even those MFA places where I can click a notification or enter a code, I still sometimes need a password.
That’s fine, and I think MFA is a good solution, but it doesn’t alleviate the need to have a strong password. Troy Hunt has written about this topic because we as a collective do a poor job of building passwords. Especially with regards to length. Some of that is poor app (and database) design where we unnecessarily limit password length. However, some of the issues are our fault, as we continue as a group to use poor habits and practices.
There are many guidelines to use with passwords, one of which is the length. The length you should use keeps increasing because hardware power keeps growing. Because of new attacks and techniques, we ought to review what we think is strong on a periodic basis. My password manager defaulted to 8 characters when I started using it over 20 years ago. Since then I’ve increased that to 12, and now 15. I ask for mixed upper case, lower, and numbers, along with symbols. These are so random that every time I need to give one to my wife to enter in, she’s annoyed with the length and mix of keys that need to be pressed.
I haven’t seen the brute force table from Hive Systems before, but I like the visual. It helps you determine how strong your password is with modern hardware. This is a similar graphic to the one I used in an encryption talk years ago, where it showed how much it would cost to rent compute power on AWS to brute force crack various algorithms. In case you were wondering, about 5 years ago you could crack a 512bit key on AWS for less than US$75.
I like the graphic, and it shows that my 15 character passwords should be safe for years. This Friday, I’m wondering if you’re comfortable with your password lengths? Are they crack-able in less than a year? Take a look at the graphic and let us know.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to stay fully present while drinking your tea or coffee.
I tried to do this last week at SQL Bits with a few people. I drink a lot of coffee at events, perhaps too much, but at times I’d get a cup and then find someone to chat with. I’d listen and focus on them for the 5-10 minutes we might talk. I’ve found it to be an interesting and helpful way to spend time with people that I see rarely, or those I’ve just met.
I want my energy to be with them, and not be distracted. I try to put my phone in a pocket, and just be with them. This doesn’t always work, but I am trying to be better.
It’s something I enjoy, and I look forward to doing more at future events.
This is part of a series on my preparation for the DP-900 exam. This is the Microsoft Azure Data Fundamentals, part of a number of certification paths. You can read various posts I’ve created as part of this learning experience.
I don’t know a lot about Power BI. I’ve lightly hacked and played with it, but it has evolved and changed so quickly that I needed to dig into some concepts.
Power BI Dashboards are a part of what you need to know for this exam. Over the years, I’ve lightly made a few reports, but not dashboards, so this was an area I needed to study up on a bit, especially these concepts.
Dashboard Basics A dashboard is different from a report. A dashboard is
- a single page
- available in the Power BI Service only (not Power BI Desktop)
- composed of tiles
- can use data from one or more reports, and more than one dataset
- one dashboard can be featured
- supports natural language queries
- can’t see the underlying data, but can export data
Tile Sources A tile can be from:
- a report (a visualization)
- another dashboard
- an Excel workbook in OneDrive for business
- Quick Insights
- An on-premises paginated report from Power BI Server or SSRS
There can be standalone tiles for images, text boxes, video, streaming data and web content.
These are a series of facts I think are important to understand about Power BI Dashboards
I don’t know if I’d be a chief, but I am glad that more companies are recognizing that data is one of the most valuable assets in most companies and starting to hire Chief Data Officers (CDO) to manage governance and security projects, as well as oversee data quality and management. With Chief Information Officers (CIO) and Chief Technology Officers (CTO), it appears that technology is becoming a force in the executive ranks.
Most of the companies I’ve worked for only had one chief-anything to do with technology. It seemed that anything to do with computing fell under that individual. Adding a CIO and, now, a CDO shows that there there is a complexity in working with computer systems. The infrastructure and architectures of systems are important, but how we capitalize on information is crucial. I see a CIO as more of a business role than a technology one.
The CDO seems to be a specialist that owns the data and ensures that it is well cared for, protected, and more importantly, managed appropriately. That would include knowing what data we have, how it is classified, how data should be protected, when to dispose of it, and how to assess the risk of keeping data around. It seems like a mix of strategic and tactical areas that a CTO or CIO might be tempted to consider low priority issues.
I think data is one of the most important assets in many companies. We are always searching for ways to better understand our environment and make decisions that improve the way our organization works. We depend on data, and these days data is often something we lean on heavily. We need to protect it, especially as more regulations appear all the time that require us to change how we use and handle data.
I don’t know how many of you have a CDO in your organization, or how quickly this will become common. I do know that the growing importance of the functions of this role are impacting more and more people who deal with data. While they might feel like a pain to adjust to, the more that we pay attention to how data is handled, the better off we all will be.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to have a no plans day and notice how that feels.
Incredibly hard for me to do this. I really, really struggle. How so? A little story.
After a volleyball tournament recently, an all day (5am wakeup, arrive by 7, busy until 330p), I got some food and drove home. I got home around 530. As I was close to home, my wife noted that the generator outside wasn’t working right and she was hoping to do a training session with a horse. Mind you, she was with me all day and then needed to go pick up this horse.
I was looking forward to unwinding at night, play a little guitar and rest. Instead, I got out there, dragged a second generator out there, hooked it up, tested lights, then dragged the old one in. I stood in the barn reading for about 20 minutes until she arrived. I explained how things were working, then went inside.
I could have told her we need to let this go for a day, but I didn’t. I had been somewhat productive coaching during the day, but it wasn’t enough. Not for her either.
This weekend I’m going to try this on Sunday. We are visiting our daughter in NY, and I’ll work Friday while she’s in class, and Sat we have plans to watch volleyball and go to a TedX talk. Sunday, however, I’m not making plans. I want to just see how things go, what my daughter wants to do and just flow with the day. No plans, no worries. If I work out, great. If not, no plans.
I was lucky to travel to London last week and attend SQL Bits 2022. This was the first live SQL Bits I’ve been to in 4 or 5 years, and the first one held since 2019. Two years away, but back strong and as wonderful as ever.
The event was at the London ExCel center, which is a large exhibition hall. There were 4 or 5 other events taking place, and the 1,000+ SQL Bits event was in less than 1/10th of the space. Still, a large space that reminded me of what a large conference is like.
The event hall was a large space, surrounded by curtains, with booths in the middle and around the edge. The Community Zone was a section near registration that had some benches, large pillows, and of course, an arcade machine. Various people were hanging out here all week.
The “rooms” were spaces that were curtained off with large screens, stages, and seating that was separated out with tables and space. Overall, I liked the room, though there was a low rumbling of crosstalk from other sessions. Not so loud that it felt like I was listening to two speakers, but more there was a lot of ambient noise, and I had to focus on my speaker. Applause was a bit loud.
I saw lots of friends. Here’s my Kevin Kline shot, adding to my collection. I’ve seen Kevin at events all over the world for the last 20 years and it’s always a treat.
I also ran into John Morehouse and many others. I didn’t end up taking as many pictures as I would have liked. A sore wrist and a phone stuck in the wrong pocket had me rarely taking it out.
The Friday keynote was a Community one, which I really liked. Rob Sewell and Ben Weissman hosted and asked a number of us to pick our favorite feature in the data platform.
I had my moment on stage and someone got a nice shot.
Both keynotes were live in one of the 12 rooms, but broadcast to the others. That meant that there wasn’t a large space where everyone had to go to in order to watch. They could go to session rooms and see the keynote. I like this idea, and would like to see it at more events. Requiring a large space is hard, and expensive.
One thing I like about Bits is lots of breaks, with time between sessions to walk around, see people, get coffee, etc. Some events seem to have everyone running from place to place, but Bits is more relaxed.
Speaking of relaxing, I was wiped out by Friday afternoon. I spent a good portion of time relaxing in the main expo hall, talking with Redgaters and others.
London is an amazing city, one of my favorites, and I had a couple nice dinners out with Brent Ozar. Here’s one with Erik Darling and his wife. Wonderful times with friends that I’ve missed the last two years.
The party Friday night was a fun affair. I liked that some food and refreshments were in the expo hall, which was quieter. A smaller space behind there was the louder party with music, games, and dancing. Not quite my style, but it was nice to see people having fun. I had to fly Saturday morning, so a quick photo with Grant and Kathi as I was leaving.
SQL Bits went too quick, and I was sorry that I wasn’t there for the last day. It was a lot of fun, and a sign that people still want to meet in person and conferences will come back. I’m looking forward to future events and hopefully SQL Bits in 2023 as well.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to eat mindfully. Appreciate the texture, taste, and smell of your food.
I don’t do this too often, especially lately. I have a plainer diet, though I am slowing down how I eat and appreciating the few things I do eat.
However, I was invited to an amazing dinner by Brent Ozar in London and I went out of my way to try and appreciate the food. I’m not a foodie, but I did enjoy the time at one of Gordon Ramey’s restaurants where the chefs are trying to show off.
This was an oyster. Looks like an oyster, but it was jellied, like a gummi bear or other jell candy. Softer, and quite amazing. I don’t love oysters, but this was something special, with the jellied taste instead of a slimy one.
Roast duck was amazing. The presentation was something, as we saw the whole duck. But then this slice smelled wonderful, and it was tender and full of the duck flavor, with a blend of spices accenting things. The side was turnips, with a turnip puree as a paste. Not a fan, usually, but this was an enjoyable bite.
The fish was great. Cod, but with caviar on it, and potatoes under. The popping of the caviar, slightly salty, with a very tender, flaky fish. Amazing.
There was more, and all of it made for a wonderfully memorable evening.
Years ago I set up an email account for my son using Gmail, periodically forwarding things I thought he might find interesting. One day I was with him at his PC and asked if he’d seen something from me. He said he hadn’t and opened his mail. He had dozens of emails, many of them marketing. I asked him why he didn’t delete some of the obvious marketing ones that he didn’t care about. He pointed to the sidebar, where the usage of his account was listed. It showed a few percent of the 10GB he was assigned in use. He said if the usage got too high, he would. For now, he would rely on search to find things.
That was fascinating to me. I’d grown up with limited space, used folders for organization, and pruned out anything old or useless. It was an eye-opening conversation to me on the difference in how generations looked at computing. That same thing is happening on a larger scale. As this piece shows, newer generations are approaching the way they use computers in a completely new way. They don’t even necessarily know how to find items on their computer by browsing, which is strange for older users. It’s a trend that vendors embrace. In Windows, Microsoft surfaces “Documents”, “Pictures”, “Music”, etc., and I find many people have no idea where those folders actually are in the file system.
I used to worry that we were losing skills that were needed. However, now I’m not so sure. The scale of items we might save has grown. In many ways, why would we care where they are on a system. Really, we need a way to access them, and linkages, search, and other techniques might be better than relying on our memories of how we’ve filed things. As I think about it, many of the problems of deploying software over the years have been because of incorrect paths. Why should developers manage that? Why don’t projects and compilers just sort this out for us? How many times have I had to add an entry to my PATH variable? Shouldn’t executable software solve that for me?
Certainly someone needs to care about locations and security and various other details, but for most people using a system, these are unnecessary details. I don’t know “where” stored procedures or functions reside. SQL Server ensures they exist somewhere and the various Object Explorers in tools put them in a place I can find.
I embrace these types of changes and encourage our industry to make installing and using software, as well as the management of files and other items easier and more autonomous.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.
I had an authorization issue with my account, and I decided to delete it and re-add it. That’s the subject for another day, but before I could delete it, I had to remove the ownership of some databases. You can’t delete a login that owns databases.
I realized I wasn’t sure how to do this, so I wrote this post.
A Deprecated Proc There used to be a dbo.sp_changedbowner proc that was used, but I know this is deprecated and it shouldn’t be used. It likely would work fine in SQL Server 2019, but I also know there should be more modern code. I decided to look, as I ought to know what is recommended these days.
In searching around MS Docs, ALTER AUTHORIZATION comes up in the list. I checked, and this allows me to transfer the ownership of a securable, which a database is one of the items in the list. Example F shows what I want to do and uses this code:
``` ALTER AUTHORIZATION ON DATABASE::dbname TO [login]
```
I can replace dbname and login with the values I need.
Which Databases? I have a lot of databases, and I don’t need to change them all, though I could. In my case, I decided to get a list of databases and owners. If you query sys.databases, there is an owner_sid column. If you join that with sys.server_principals, you can do so on the SID column. This query shows me what I need:
SELECT d.[name], sp.[name] FROM sys.databases d
INNER JOIN sys.server\_principals AS sp
ON d.owner\_sid = sp.sid
The results are here:
In some sense I hate that “sa” isn’t the default owner, but I get it. There might be a need for other accounts. However, my account is a sysadmin, so my view here is that “sa” ought to be listed.
I digress. Now that I have a list, I can limit it to my account with a WHERE clause. I can take that list of items and build the code. I could use a cursor, but this is a one-off task, so this works:
```
SELECT
'ALTER AUTHORIZATION ON database::' + d.[name] + ' TO sa;'
, d.[name]
, sp.[name]
FROM
sys.databases d
INNER JOIN sys.server_principals AS sp
ON d.owner_sid = sp.sid
WHERE sp.name = 'ARISTOTLE\Steve';
GO ```
This gives me the code in the results I want to run. I copy paste this and I have a bunch of statements to run.
Despite Grammarly not being happy, this worked fine.
SQL New Blogger As soon as I realized I needed to do this, I knew there were two posts here. One on the removal and adding back of my Windows account, and the second on this topic (when the first didn’t work).
This took about 15 minutes extra, finding the docs and writing some code, but it’s a good example of where a small situation that occurred helped me find something to write about. Easy for you to take little tasks like this and document your knowledge when you learn something.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to take three calming breaths at regular intervals during the day.
This is an easy one for me. Since I’ve been practicing yoga for 10-12 years, I often find myself taking more conscious breaths at times. Fill up deeply, hold briefly, and slowly let them out.
It’s amazing how calming 3-4 deep breaths can be.
This is part of a series that covers my experience with a Tesla Model Y.
I was driving home from a ski trip recently on I-70 E in Colorado. My wife wanted to call my daughter and went to switch the connected Bluetooth device from my phone to hers. She connected her phone and made the call, but she couldn’t get the audio to come out of the car speakers. She ended up talking directly on the phone while trying to disconnect and reconnect a few times.
At some point my wife finished her call and went to click the icon on the Tesla screen to disconnect and nothing worked. She tried pressing the car menu button, the spotify icon, even the climate controls. Nothing worked.
At this point I was driving 65mph on a busy highway, with snow on the side shoulders. Not the place to worry.
I knew the car should keep running, but there is something a little uneasy about not being able to change any settings while driving down the road. We debated what to do for a few minutes. I knew there was a reset procedure, but I wasn’t interested in testing it while driving.
We kept going down the highway, planning on taking an exit and then rebooting. As my wife searched for information online as to what to do, the screen actually restarted itself. Not the full restart I’ve seen with software updates, but apparently a soft restart of the UI. After 10-15s after the screen went dark, I had a working UI again.
Hard and Soft Reboots We looked up the procedure and there are both hard and soft reboots of the car. A soft reboot can be done in the Model Y by holding both steering wheel buttons down until the car resets. This should only be done while the car is stopped, which is what I expect.
A hard reset, and power cycling the car, comes from the UI itself. You should allow the car to be powered down for 2+ minutes before doing anything.
In general, I’m not overly worried about the UI rebooting, but I do think that this is a case where a secondary screen, running an independent OS, would be good to keep the driver informed of at least speed.
Am I Concerned? Not now. The car seems to work independently of the UI, which is what I would hope was the case. It’s lightly disturbing, but that’s about it. Now I know what to expect.
Software Updates Usually there’s a notification in my app of a software update. It also appears in the UI. I used the UI to do one manually, watching the screen give status info similar to how a PC or mobile does. Very uninteresting.
For all of them since the first one, I usually just approve the update in the app at night. The car is plugged in and I haven’t worried about watching it. I’ll check in the am, and I haven’t had issues.
I do see UI elements moved, which is slightly annoying, but not critical. Mostly I watch the release notes would better use images to describe movement of stuff so I don’t have to hunt around. It took me 3 or 4 minutes to find my steering wheel heater after one update, which was annoying and distracting as I kept looking at various stop signs during a drive.
This is part of a series that covers my experience with a Tesla Model Y.
After a bit over 4 months, we finally had a long road trip in the Tesla Model Y. As I live in Colorado, we went to the mountains to ski. This post covers two trips, a day trip and a multi-day trip.
Day Trip My wife and I took a day off and drove to the Keystone on a Thursday for a day ski trip. I charged the Tesla up to 93% overnight and we headed up on a snowy day. A fairly heavy snow was falling as we left the house and kept falling until we were well into the Colorado mountains on I-70.
The snow tires worked well, and I felt the car was gripping well on the road. Traffic wasn’t too bad, and the drive went as smoothly as it had gone in other cars.
The one place where I was careful was driving through Dillon, CO, where the roads were snow-packed, and there are some substantial curves. I was careful here, as the Model Y is a heavy car and slips a bit more near the bottom
A few stats from the drive up (109 miles)
- Power: 48% of the charge (93%->45%)
- Cost: $4.81 (37kWh * $0.13)
The drive down:
- Power: 38%
- Cost: $3.64 (28kWh)
As a comparison, 110 miles in the X5 costs about $20 (at $3.50/gal). The Prius would have been about $9.20 or so.
We skied most of the morning in about 20-25F weather. We then left and headed home, having lots about 2% of charge while the car was sitting in the parking lot. We drove down to Idaho Springs, and at around 28%, we stopped at the Supercharger to add some power.
We spent about 10-15 minutes adding power (about 25%). We sat in the car, checked some email and chatted, though this location is walking distance from a number of restaurants in Idaho Springs. On another day we might have walked over to have a quick lunch and left the car, as many others did.
It was an easy trip and a pleasant experience for our first trip into the mountains.
A (Cold) Three Day Trip The next week we went back, staying up at Keystone for three days. I charged the car up to 87% and we drove to coach kids and then left from the gym for a late night drive up. It was cold, around 20F, and we again used about 50% of the power. In this case, 87->33% with a few stops along the way.
The next morning, we were down to 26% on a night that dropped to about 8F. That was a 7% drop overnight. An interesting data point.
We drove over and skied, and then went to the Silverthorne Supercharger, arriving with 18% power left. Our plan was to get a cup of Starbucks while it charged, as the chargers are in the parking lot for Starbucks, but the store was closed. Not enough staff with a few out sick..
Not a problem. Rather than drive over to another place, we just charged and talked with our kids on the phone. Interesting, I pulled into charger 4B and plugged in. The car would say “starting to charge” and after 2-3 minutes it would say that something was wrong and I needed to unplug and plug it back in. I did that twice, and got the message.
I switched to a different flow and things worked flawlessly. The charger immediately jumped to about 50Kwh, putting around 240miles/hour of charge into the car. We spent about 25 minutes there, getting up to 75% before I decided to stop. I was hungry, so we left.
Overnight the temps were close to 0F, and we dropped about 15%of charge that night. The next night was –8F when I woke up, and we’d dropped about 18% of charge. Not insignificant, and I can see why people recommend waking up and charging in the am before a long trip.
If we were planning on a long drive, I’d have gotten up, gotten coffee and then gone and parked at the supercharger to eat, work a bit, and fill up. Slightly more cumbersome than a gas station, but really, I’d be moving some work time into the car rather than sitting inside and then hitting a gas station.
Overall, the car worked well under cold, winter conditions. We didn’t have any range issues, and easily charged up at the Superchargers along our route. I could pre-heat the car before driving, and I’m getting used to remembering to do this. Most of the time I think about this about 10 minutes before we pack up and go, which isn’t inconvenient. It certainly is nice to pre-heat the car as I’m paying a bill in a restaurant and have the heat going when we get in the car.
Tires worked well and gripped in the snowy conditions. It was informative to see how the battery faired driving into the mountains, as well as overnight in some of the coldest conditions I’ve seen in years. Not a lot of battery/range anxiety and charging was smooth and easy. Taking 15-20 minutes with a cup of coffee and my wife wasn’t a big interruption to our lives, and we didn’t mind taking a few breaks during our short holiday.
I ran across an article, titled When to use CHAR, VARCHAR, or VARCHAR(MAX), which struck me as something I never do. I mean, I do use varchar (and nvarchar), but I can’t remember the last time I actually created a char column. The article is worth a read, and it sets the stage for you to think about your database design process and the choices you make.
It’s Friday, and if you take a few minutes and think about the last few times you’ve added columns to a table have you used CHAR as a data type? Or do you default to varchar of any size as a general rule?
I tend to do a lot of demo work, and I help customers with different situations. In many cases, we are storing text data, often not strongly typed data. As a result, I find most customers using varchar (or nvarchar), and I’ve built the habit of using the variable structures in proofs-of-concept and demos. I find it especially handy when someone asks me to enter some data they use and then show how it would be handled.
Early in my career, I’d often tag a zipcode as a 5 character field, or a state as a 2 character abbreviation. However, these days a postal code can be a 5+4, which is 9 characters or even 10 with the plus. Many companies work overseas and may want to account for longer postal codes. States (or regions), can often be 3 characters, but sometimes more. Often we just leave 10 characters for region abbreviations (or longer) as the data might be spelled out or need to accommodate something unknown.
For many business applications, it seems that there might be a definition for what the data should be, but since exceptions can abound, often using a variable-length data type just prevents issues in the future. Add that to the fact that often we are dealing with cheap storage, and it doesn’t seem worth the time to try and get the exact size correct. Even when knowing an invoice uses 10 characters, are you sure that you won’t exceed the ten-character width? What if you acquire a company that uses 12 character invoice numbers? Easier to set this to a variable 20 and move on.
If you work in data warehousing, then you might know what your data sources contain and be more likely to choose fixed types, but is the space savings worth the work in the event that source systems change? I don’t know. I tend to plan for sources to change and allow a little padding in my schema. You might feel differently, but are the space savings worth the potential hassles in the future? I’d be curious what you think today.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to thank someone for cheering you up in the last month.
Someone didn’t actually cheer me up. Well, certainly some people did, but it was more the memory of someone made me smile and brought a moment of joy. Actually, two people.
I’ve been to Australia twice and New Zealand once. Recently my wife and I were talking about the trip we took together down there, how much we enjoyed it, and some of the people she met (I’d known them previously). The memory of standing next to this guy made me smile:
Hamish’s joy and love for life is infectious, and I miss him. I am hoping New Zealand will allow visitors again and I can return.
The same week, Martin Catherall reached out on another matter, and my wife and I enjoyed time with him in Australia. Martin also has an infectious, happy demeanor, and a wonderful accent, that I enjoy every time we talk. Hoping to get back to Australia again as well.
Thanks to both of them for cheering my up this month.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to share what you are feeling with someone you really trust.
This is tough for me. I tend to be a little closed off and private in many ways, but I’ve learned to open up a bit. Part of this is growth, part is being a good partner, part is modeling what I hope my kids to better than me.
My wife is my partner. In many ways, she is the only person I “choose” in my life that is a constant. I didn’t choose my parents, I didn’t choose my kids, and I certainly don’t get to choose co-workers. I choose friends, but there often isn’t quite the same closeness.
I won’t share my feelings here, but I have been working to let my wife know more often when I am feeling when I feel it. It’s that last part I can struggle with. I brood, I think, I over-analyze.
I’m not a stream-of-consciousness talked, but I am learning to do that more often. It’s been a 2 decade plus long journey that I continue today.
This is part of a series on my preparation for the DP-900 exam. This is the Microsoft Azure Data Fundamentals, part of a number of certification paths. You can read various posts I’ve created as part of this learning experience.
There are types of schemas the exist in data warehouses. This topic is definitely on the exam.
OLTP/Relational The type of schema that many of us work with is the standard OLTP or relational model. We have lots of transaction tables, most should have a PK, some of which have PKs. The schema expands to meet different needs and can have lots of entities.
As an example, here is a view of the AdventureWorks database.
This isn’t fundamentally different from the schema types below, but there isn’t a central, or two central, tables here. Instead, we have a lot of different groups of tables. The structure is designed for normalization, and usually has lots of tables compared to a data warehouse.
Star The star schema is often used in data warehouses. The name comes from the fact that the table arrangement looks similar to a star. There is a central fact table, which has some details of the main data, often something like sales, and a lot of foreign keys (FK). The fact_sales_order is the fact table below.
Then there are supporting tables around the fact table, linked by the FKs. These are the dimension tables, and contain details about a specific dimension or area. In the image below, we have date, employee, store, and other dimensions tables.
This is a somewhat de-normalized structure, as the primary purpose is to report on a set of facts.
Snowflake This schema builds on the star schema. Here there is still one fact table (Sales below), but the dimension tables have their own dimension tables, providing more details. Essentially, the dimension tables are normalized. An example is the Employee dimension, which has a linked Department dimension.
Galaxy The galaxy schema expands with a second fact table. In the image below, we have the sales and purchase as fact tables. There are dimension tables, which can be linked to one or more fact tables.
It isn’t that important you know how to build these schemas or design the entities for the DP-900 exam, but you do need to recognize the structures.
The cloud for computing is a fascinating structure. I know there are plenty of jokes about the cloud just being someone else’s computer, and there are good reasons not to use the cloud. However, there are also lots of good reasons to use the cloud. Whether you choose to embrace it or avoid it, cloud computing is going to be a part of our careers for a long time. The use is growing, and more and more companies are shifting workloads to cloud services.
How that will evolve, especially for data-intensive systems, will be fascinating. One of the interesting changes that seems to be taking places is the growth of database and data stores that fulfill specialized roles for customers. Snowflake might be one of the most well-known examples, but there are plenty more. The number of offerings is growing, and perhaps this is another evolution of how the cloud will integrate into more businesses.
To date, most of the large cloud providers (Azure, AWS, GCP) are offering a full stack of different systems that you use to deploy code and run applications. These cobbled-together services and platforms often lock customers into a particular cloud, though that isn’t what many organizations would like. Especially in some regulated industries that mandate multiple clouds be used for redundancy. This article talks about the cloud as a foundation on which other customers can build services or platforms, especially data platforms.
This is something I would like to see. Snowflake is something that a company can run on AWS, Azure, or GCP. CockroachDB is another that allows customers to work with the platform on the provider of their choice. I like the idea of a wider set of platforms built upon cloud providers, but in a way that allows customers to move if need be, and also pressures cloud providers to keep pricing in line with each other.
In my mind, the more we find innovative companies building cloud-native data stores and other products, the more pressure on existing companies to improve their offerings. Competition is good, and it brings us new tools, while also forcing existing companies to update their offerings. It’s not perfect, as sometimes we get new features without quality improvements in the base product, but without competition, we might not get that anyway.
I am fascinated and pleased by the cloud. The more I learn and work with it, the more I appreciate this as a truly new way of approaching the building and operating of software.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to show an active interest by asking questions about others when talking with them.
I like trying to engage with people in conversation. I don’t do it a lot, as I tend to be fairly introverted, but I have learned that when I decide to talk or listen to someone, I need to pay attention and focus.
I did this with one of the kids I coach recently. We had a few minutes, and I asked her how a school thing was going. She had said something the week before about a challenging time, so I followed up, asked how it went, and a few other questions based on her answers.
Reminds me of them times when my kids had really worked hard on something. This young lady detailed some challenges and how hard things were, but the enthusiasm and passion in her voice showed that she was proud of her work, despite the complaints she voiced.
A few years ago Ed Leighton-Dick started the #SQLNewBlogger challenge. He asked people to start writing about their career and building their own brand. I thought it was a great idea and I’ve been continuing the challenge on my own blog, writing posts and also including a short note on the effort for me to produce posts.
It’s early 2022, and if you haven’t been blogging, and you want to grow your career, start writing. Set up a blog and produce some posts. You will build communication skills, you will showcase your knowledge, and provide potential employers with some due diligence about why you would be a good addition to their team.
Think about tagging these posts with #SQLNewBlogger as well. You’ll promote these alongside other people’s posts and perhaps will find yourself a new, better position this year.
I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.
Today’s tip is to send someone a message to let them know you’re thinking of them.
This is an easy one for me. I do try to reach out to some friends, but I took a few moments to send a couple messages and emails to friends, letting them know I’m thinking of them.
This tip never fails to help me feel better about the world when I hear from a friend.
This is part of a series on my preparation for the DP-900 exam. This is the Microsoft Azure Data Fundamentals, part of a number of certification paths. You can read various posts I’ve created as part of this learning experience.
I passed the DP-900 exam. I did well, though I likely over-prepared.
I have typically been a good test taker, but I’m nervous about them, especially when they cost money. I did well on most finals in college, but was always very nervous. For most of these exams, even though my employers have paid, I’ve been worried.
This post looks back at the things I did, building on my prep post.
Final Prep Since this was for work, and they’d pay for it, and because I wanted to review things, I got access to the Official Microsoft Practice Test. This is US$99 for 30 days or a bit more for 60 days.
Side note: I once worked for an education company in the early 2000s that partnered with MeasureUp and I had access to these tests.
I purchased this about a week before I thought I’d take the test. I’d been lightly going through the courses and reading docs on and off for a few weeks, usually about an hour every few days. I took the first practice test, which simulates the exam (49 questions, timed) and got a 650. Not good enough.
I went back through some of the self-paced training quicker, and then I dug into books online in a more intense fashion, for about an hour a day for a few days. I then retook the practice test and got to a 780. Passing, but not great.
I consistently struggled with CosmosDB and Synapse, and a few Power BI and Azure Storage items, so I concentrated on those items.
The week I’d orginally planned to take the test was spent taking a practice test every day, and then using the mistakes I’d made to focus on some new concepts. Once I passed 2 days in a row, I scheduled the exam. I spent the next 4 days taking practice tests and studying.
One of the best things I did was get a notebook and a pen. I then watched John Savill’s Exam Cram and took notes by hand. I find that helps me remember things. I paused in a few areas if I wasn’t sure I knew what he was talking about and looked up details in MS Docs.
The morning of the exam I went to the gym, which helps me relax. I then got a cup of coffee and took the practice test, but every question. All 150. I scored a 970, and thought I was ready.
The Actual Exam While you can take this at home in a room by yourself, I wasn’t confident I could get by without someone interrupting me, or something going sideways at the ranch. I get interrupted regularly as people forget I’m working or don’t realize I’m on a call. I scheduled this at a local test center that I’ve used for years.
If you haven’t been in a test center, it’s usually fairly strict. This one does FAA/Pilot testing, so they are very careful. The things to know:
- Nothing goes in to the exam room with you (except mints).
- You need an ID, they take a picture, have you digitally sign an MS doc, and paper sign a couple waivers as well.
- Mask the whole time, which isn’t great with my readers
- ID, watch, phone, even lip balm all get locked up. I take the key into the room.
- I grabbed a few extra mints because I’m nervous.
- There are dry erase markers and pens for notes, and ear muffs for quiet. Monitor, keyboard and mouse on the desk, cube partitions separating everyone and cameras above. In my center there are 8 desks and at least 3 cameras watching you.
- I’ve never taken a break, but I also know to use the bathroom before going in (again, nervous). I did have an issue once, raised my hand, and within 3-4 minutes someone came in to check on me (computer froze).
Exam Format The format is this:
- 3 minute survey of your experience (how confident are you about azure rdbms, analytics, etc.)
- 45 minutes for the exam, 49 questions. You can mark questions for review and then a list of these appears at the end that you can go back and change answers.
- A mix of multiple choice (radio buttons), select x (2 or 3 answers required), lots of “click yes for true, no for false”, a few drag and drop from a list of answers.
- No case studies, no free text entry
- 3-4 minutes for feedback on specific questions. I thought a few were strangely worded, so I left feedback on some.
The practice test mirrors this well.
Exam Coverage I can’t disclose the questions, but I can give you a few things to think about. Note I got 49 of who knows how many questions. I would guess there is a pool of a few hundred, but I don’t know.
If you look at the skills measured, I would say this about the major areas:
- Describe core concepts – I had at last 3, maybe as many as 7 questions directly in this area. These are fairly easy to answer if you know what these are. I’ll do a couple prep posts in these areas.
- Describe working with Relational Data – Again, I find the workload stuff fairly easy, and I had 2-3 questions here. The PaaS, IaaS stuff is important and I’ll post things that matter. The understanding of the basics of Azure SQL family, Synapse, and how you might “accidentally” work with PostgreSQL/MySQL/MariaBB matter. There also were some light questions on concepts around tools, connectivity, firewalls, and CLIs. Again, I’ll post prep. The T-SQL specifics were easy, a couple questions there.
- Describe work with non-relational data – This is a weak area for me.I think some of this is not being completely confident on how/where I use different NoSQL structures, and the nomenclature being strange. Knowing the types of data stores and when to use them matters. Practice test helped here. You do need to know a good outline of CosmosDB and the Azure Storage structures. Glad I spent time here.
- Describe Analytics workload – Also a weak area. I don’t know Synapse well, and I think the docs are poor. They tend to be written, in my opinion, assuming you know some things. They don’t describe things well, and to be fair, Synapse appears to have evolved a lot and quickly. You need to know a bit about when/where you use tools, understand some concepts about ADLS Gen 2, Databricks, Synapse, and HD Insight. A lot, but this is high level, not in the details. Know conceptually where you use ADF with these tools. The PowerBI stuff means you better know the What is Power BI? and the Basic Concepts docs well. Again, I’ll do a prep post.
My Recommendations First, make sure you could explain each concept in the skills document to a friend, every single line, with about 3-4 minutes of talking. Not a highlight, but that you’d sound like you had a grasp of each area. If not, dig into docs.
Second, if you want a good outline of data services on Azure, this helps you focus. You need to know the storage stuff, the relational options, and the analytics stuff. I felt like I learned a lot in about 3 weeks of prep. Not a ton each week, but regular. Really this was about 10 days of constant prep, something every day, and a few weeks of sporadic prep.
Learning how to dig into and get a level of detail was tough, but the practice test was worth the $$$ to me. It helped me focus. Jon’s cram video, about 90 minutes, was a good place to start. I wish I’d done that first, taken notes, and then looked up weak areas.
Hard to determine weak areas without testing. Again, practice tests help. I can’t mimic the exam without worrying about NDA, but I’ll try to give you a set of things to know in various posts.
Work with a friend. I’m thinking to do some prep classes with my local user group. If you’re interested, let me know. Maybe I’ll just do some online recordings. Other people helped me, and I can give you a set of things to learn.
Lastly, over prepare a bit. Don’t spend months, but think of this like a final exam from high school. Learn some things, practice a bit with the online stuff, and then increase your focus as the exam date approaches.
If you have questions, ask. No specific questions or answers disclosed, but I’ll try to help where I can.