325. What Data Should We Log/Trace And What Should We Ignore

Logging lets us know where the problems Logging lets us know where the problems are happening. It also lets us see how are happening. It also lets us see how are happening. It also lets us see how our app is being used. But if you aren't our app is being used. But if you aren't our app is being used. But if you aren't careful, logging can actually make it careful, logging can actually make it careful, logging can actually make it harder to understand where the problems harder to understand where the problems harder to understand where the problems are occurring and how your application are occurring and how your application are occurring and how your application is being used. So let's talk about how is being used. So let's talk about how is being used. So let's talk about how to properly implement logging in a way to properly implement logging in a way to properly implement logging in a way that will actually benefit our that will actually benefit our that will actually benefit our application in today's episode of Dev application in today's episode of Dev application in today's episode of Dev Questions. Software development is more than just Software development is more than just writing code. So let's talk about the writing code. So let's talk about the writing code. So let's talk about the rest of it. Specifically, let's talk rest of it. Specifically, let's talk rest of it. Specifically, let's talk about how to properly implement logging. about how to properly implement logging. about how to properly implement logging. And let's start with an illustration. And let's start with an illustration. And let's start with an illustration. Back when I was a very new IT director Back when I was a very new IT director Back when I was a very new IT director at a small college, I was working with at a small college, I was working with at a small college, I was working with my network administrator to implement my network administrator to implement my network administrator to implement server logging. Up until that point we server logging. Up until that point we server logging. Up until that point we didn't have any central logging errors didn't have any central logging errors didn't have any central logging errors were discovered as the by the users or were discovered as the by the users or were discovered as the by the users or you know but we stumbled across them or you know but we stumbled across them or you know but we stumbled across them or maybe if we looked through the event maybe if we looked through the event maybe if we looked through the event viewer and we had a full network with viewer and we had a full network with viewer and we had a full network with active directory and exchange server and active directory and exchange server and active directory and exchange server and SQL server and file servers and all the SQL server and file servers and all the SQL server and file servers and all the rest. So we we found this software that rest. So we we found this software that rest. So we we found this software that would centrally manage all these logs.

would centrally manage all these logs. would centrally manage all these logs. It would would track the different It would would track the different It would would track the different things going on our servers. it would things going on our servers. it would things going on our servers. it would make sure that it looked out for things make sure that it looked out for things make sure that it looked out for things like hey you got low disc space on this like hey you got low disc space on this like hey you got low disc space on this server or hey this server is hitting a server or hey this server is hitting a server or hey this server is hitting a very high CPU all day long or you know very high CPU all day long or you know very high CPU all day long or you know we've got a bottleneck in memory here or we've got a bottleneck in memory here or we've got a bottleneck in memory here or or whatever the case may be. We're or whatever the case may be. We're or whatever the case may be. We're throwing exceptions or whatever. So this throwing exceptions or whatever. So this throwing exceptions or whatever. So this software was was great and it could do software was was great and it could do software was was great and it could do so much stuff. We went to training. We so much stuff. We went to training. We so much stuff. We went to training. We we figured out how to make the whole we figured out how to make the whole we figured out how to make the whole thing work correctly, how to get the thing work correctly, how to get the thing work correctly, how to get the most out of the software, and then we most out of the software, and then we most out of the software, and then we implemented it in I think it was like 20 implemented it in I think it was like 20 implemented it in I think it was like 20 servers. Um, and so we we implemented servers. Um, and so we we implemented servers. Um, and so we we implemented all these different servers. We had it all these different servers. We had it all these different servers. We had it we set it up and configured. So we make we set it up and configured. So we make we set it up and configured. So we make sure we looked at all the different sure we looked at all the different sure we looked at all the different errors, all the different issues and errors, all the different issues and errors, all the different issues and made sure that we were tracking all the made sure that we were tracking all the made sure that we were tracking all the things we needed to in order to have a things we needed to in order to have a things we needed to in order to have a really good bird's eyee view of what's really good bird's eyee view of what's really good bird's eyee view of what's going on with the entire network. going on with the entire network. going on with the entire network. and we barely ever used it. And here's and we barely ever used it. And here's and we barely ever used it. And here's why. Because the first day, my network why. Because the first day, my network why. Because the first day, my network administrator was getting the emails for administrator was getting the emails for administrator was getting the emails for all of the errors, all the all the big all of the errors, all the all the big all of the errors, all the all the big problems he'd get the emails for. And problems he'd get the emails for. And problems he'd get the emails for. And his inbox was flooded to the point that his inbox was flooded to the point that his inbox was flooded to the point that he created a rule that said, "Hey, if he created a rule that said, "Hey, if he created a rule that said, "Hey, if it's coming from the logging server, put it's coming from the logging server, put it's coming from the logging server, put in a different folder because I can't in a different folder because I can't in a different folder because I can't even see my normal emails."

even see my normal emails." even see my normal emails." Then what do you think happened? At Then what do you think happened? At Then what do you think happened? At first we started going through those first we started going through those first we started going through those those emails but then some of them were those emails but then some of them were those emails but then some of them were like hey that that you know that server like hey that that you know that server like hey that that you know that server is getting low on disc space but we knew is getting low on disc space but we knew is getting low on disc space but we knew that and that's not so we're going to that and that's not so we're going to that and that's not so we're going to address or hey that server is kind of address or hey that server is kind of address or hey that server is kind of you know going at 80% plus of CPU usage you know going at 80% plus of CPU usage you know going at 80% plus of CPU usage but that's a good thing we want it to be but that's a good thing we want it to be but that's a good thing we want it to be there and so we started seeing all these there and so we started seeing all these there and so we started seeing all these issues where for some servers that would issues where for some servers that would issues where for some servers that would have been an issue but for others it's have been an issue but for others it's have been an issue but for others it's not and so we started ignoring issues not and so we started ignoring issues not and so we started ignoring issues but then we had a hard time finding the but then we had a hard time finding the but then we had a hard time finding the issue that were really issues. issue that were really issues. issue that were really issues. So what we found out is just because an So what we found out is just because an So what we found out is just because an issue seems critical in a vacuum doesn't issue seems critical in a vacuum doesn't issue seems critical in a vacuum doesn't mean it actually is. And if you're not mean it actually is. And if you're not mean it actually is. And if you're not going to do something about it, then going to do something about it, then going to do something about it, then it's not actually an issue. So either it's not actually an issue. So either it's not actually an issue. So either critical alerts need to be dealt with or critical alerts need to be dealt with or critical alerts need to be dealt with or they need to be downgraded. Otherwise, they need to be downgraded. Otherwise, they need to be downgraded. Otherwise, you drown out the real critical alerts. you drown out the real critical alerts. you drown out the real critical alerts. And that's really hard to do because And that's really hard to do because And that's really hard to do because when you're looking at things, you're when you're looking at things, you're when you're looking at things, you're saying that is an issue. But if you're saying that is an issue. But if you're saying that is an issue. But if you're not gonna deal with it, is it? I mean, not gonna deal with it, is it? I mean, not gonna deal with it, is it? I mean, it may be something you want to address.

it may be something you want to address. it may be something you want to address. It may be on your I would love to take It may be on your I would love to take It may be on your I would love to take care of that list, but unless it's a care of that list, but unless it's a care of that list, but unless it's a issue you're going to say, yes, I'm issue you're going to say, yes, I'm issue you're going to say, yes, I'm going to address that now. It's not going to address that now. It's not going to address that now. It's not really something you should have in your really something you should have in your really something you should have in your critical alert list. And so what critical alert list. And so what critical alert list. And so what happened was his inbox was constantly happened was his inbox was constantly happened was his inbox was constantly full of alerts. And we just never seem full of alerts. And we just never seem full of alerts. And we just never seem to do a good job of figuring out which to do a good job of figuring out which to do a good job of figuring out which ones are critical, which ones aren't. ones are critical, which ones aren't. ones are critical, which ones aren't. And so, yes, we found a problem when a And so, yes, we found a problem when a And so, yes, we found a problem when a user reported something. We go to the user reported something. We go to the user reported something. We go to the log server and go, "Oh, yep, that issue log server and go, "Oh, yep, that issue log server and go, "Oh, yep, that issue was there. We got the alert for it 18 was there. We got the alert for it 18 was there. We got the alert for it 18 times, but it got lost in the middle of times, but it got lost in the middle of times, but it got lost in the middle of the thousands of other alerts that were the thousands of other alerts that were the thousands of other alerts that were going off." And so just because those going off." And so just because those going off." And so just because those things seemed like they should be logged things seemed like they should be logged things seemed like they should be logged doesn't mean they should have been. We doesn't mean they should have been. We doesn't mean they should have been. We should have been logging just what we're should have been logging just what we're should have been logging just what we're going to fix. So let's talk about how to going to fix. So let's talk about how to going to fix. So let's talk about how to set up logging that works. Now we're set up logging that works. Now we're set up logging that works. Now we're talking more about logging inside of talking more about logging inside of talking more about logging inside of your application. So this is not exactly your application. So this is not exactly your application. So this is not exactly about, you know, server logs, but they about, you know, server logs, but they about, you know, server logs, but they the things still apply. So let's talk the things still apply. So let's talk the things still apply. So let's talk about how to set up the login that about how to set up the login that about how to set up the login that actually works for you. Number one, only actually works for you. Number one, only actually works for you. Number one, only mark issues as critical if you mark issues as critical if you mark issues as critical if you absolutely need to do something about absolutely need to do something about absolutely need to do something about them.

them. them. Too many people say, "Well, that's a big Too many people say, "Well, that's a big Too many people say, "Well, that's a big deal." Is it, though? Again, if it's a deal." Is it, though? Again, if it's a deal." Is it, though? Again, if it's a big deal, but you don't do anything big deal, but you don't do anything big deal, but you don't do anything about it, it's not a big deal. It that's about it, it's not a big deal. It that's about it, it's not a big deal. It that's just information at that point. If just information at that point. If just information at that point. If you're not going to act on it, there's you're not going to act on it, there's you're not going to act on it, there's no reason to make it a critical issue. no reason to make it a critical issue. no reason to make it a critical issue. make it anformational issue. And yes, I make it anformational issue. And yes, I make it anformational issue. And yes, I know sometimes you may go, "But that's know sometimes you may go, "But that's know sometimes you may go, "But that's that's that's important." that's that's important." that's that's important." It's only important if you do something It's only important if you do something It's only important if you do something about it. That's really what it comes about it. That's really what it comes about it. That's really what it comes down to. It doesn't matter what you say, down to. It doesn't matter what you say, down to. It doesn't matter what you say, it's what you do. This is a fact for all it's what you do. This is a fact for all it's what you do. This is a fact for all of life. If it doesn't matter what you of life. If it doesn't matter what you of life. If it doesn't matter what you say, it matters what you do. So if you say, it matters what you do. So if you say, it matters what you do. So if you say this is critical, but then you do say this is critical, but then you do say this is critical, but then you do nothing, it's not critical. So, be very nothing, it's not critical. So, be very nothing, it's not critical. So, be very careful to only careful to only careful to only bubble up things that are critical, bubble up things that are critical, bubble up things that are critical, things that you're actually going to do things that you're actually going to do things that you're actually going to do something about. Number two, only log something about. Number two, only log something about. Number two, only log errors as errors if they're unexpected errors as errors if they're unexpected errors as errors if they're unexpected or need to be addressed, such as, you or need to be addressed, such as, you or need to be addressed, such as, you know, like, hey, this this service has know, like, hey, this this service has know, like, hey, this this service has crashed. Okay, that needs to be crashed. Okay, that needs to be crashed. Okay, that needs to be addressed and that's that's an error addressed and that's that's an error addressed and that's that's an error that need to be addressed. Um, if that need to be addressed. Um, if that need to be addressed. Um, if there's an exception that happens, but there's an exception that happens, but there's an exception that happens, but it's not something we're going to it's not something we're going to it's not something we're going to address, then don't log it as an error.

address, then don't log it as an error. address, then don't log it as an error. Um, even though the error logs are the Um, even though the error logs are the Um, even though the error logs are the ones that take the exceptions. And you ones that take the exceptions. And you ones that take the exceptions. And you might go, well, I have to put them might go, well, I have to put them might go, well, I have to put them there. No, you don't. So, you don't need there. No, you don't. So, you don't need there. No, you don't. So, you don't need to log the network connection was went to log the network connection was went to log the network connection was went down for the for the user. You don't down for the for the user. You don't down for the for the user. You don't need to log that unless you need to need to log that unless you need to need to log that unless you need to know, oh, the network is going down in know, oh, the network is going down in know, oh, the network is going down in this area. For example, if you have this area. For example, if you have this area. For example, if you have microservices and two different microservices and two different microservices and two different microservices, maybe one's an API and microservices, maybe one's an API and microservices, maybe one's an API and the other one's talking to the API, the other one's talking to the API, the other one's talking to the API, maybe that network connection is all maybe that network connection is all maybe that network connection is all internal and if that goes down, yeah, internal and if that goes down, yeah, internal and if that goes down, yeah, you should know about that and that you should know about that and that you should know about that and that would be something critical. But if the would be something critical. But if the would be something critical. But if the user is on their laptop and they go user is on their laptop and they go user is on their laptop and they go through a tunnel and they lose the through a tunnel and they lose the through a tunnel and they lose the connection to the cell tower and it connection to the cell tower and it connection to the cell tower and it comes back up afterwards, that's not an comes back up afterwards, that's not an comes back up afterwards, that's not an issue that you need to be logging. So issue that you need to be logging. So issue that you need to be logging. So even though it might throw an exception even though it might throw an exception even though it might throw an exception on the user's machine, even though it on the user's machine, even though it on the user's machine, even though it might be triggered as an error, you need might be triggered as an error, you need might be triggered as an error, you need to go that's not really an error. And so to go that's not really an error. And so to go that's not really an error. And so this is where being granular about this is where being granular about this is where being granular about catching these exceptions and going this catching these exceptions and going this catching these exceptions and going this one's important, this one's not is one's important, this one's not is one's important, this one's not is important. If you're doing a a try catch important. If you're doing a a try catch important. If you're doing a a try catch and it's catching ex, you know, the ex and it's catching ex, you know, the ex and it's catching ex, you know, the ex the raw exception ex uh where you're not the raw exception ex uh where you're not the raw exception ex uh where you're not specifying which exceptions and you're specifying which exceptions and you're specifying which exceptions and you're just saying, "Hey, make every one of just saying, "Hey, make every one of just saying, "Hey, make every one of these an error." That's not very these an error." That's not very these an error." That's not very efficient. It's going to clog up your efficient. It's going to clog up your efficient. It's going to clog up your errors in your logs. So maybe at first errors in your logs. So maybe at first errors in your logs. So maybe at first you do that, but then figure out, oh, you do that, but then figure out, oh, you do that, but then figure out, oh, we're getting these network drop offs we're getting these network drop offs we're getting these network drop offs and those network drop offs aren't and those network drop offs aren't and those network drop offs aren't really a big deal. Well, then in your

really a big deal. Well, then in your really a big deal. Well, then in your catch, have another catch before your catch, have another catch before your catch, have another catch before your your generic exception exception your generic exception exception your generic exception exception or whatever it is. That's the actual or whatever it is. That's the actual or whatever it is. That's the actual network error. Catch that one and maybe network error. Catch that one and maybe network error. Catch that one and maybe let the user know, hey, you don't have a let the user know, hey, you don't have a let the user know, hey, you don't have a network connection right now. But don't network connection right now. But don't network connection right now. But don't don't log that. There's no reason to log don't log that. There's no reason to log don't log that. There's no reason to log that if you're not going to do something that if you're not going to do something that if you're not going to do something with it. with it. with it. Number three, only log warnings that are Number three, only log warnings that are Number three, only log warnings that are precursors to actual errors or critical precursors to actual errors or critical precursors to actual errors or critical issues. If there's something that is a issues. If there's something that is a issues. If there's something that is a an indicator that something is going to an indicator that something is going to an indicator that something is going to happen in the future and it will be a happen in the future and it will be a happen in the future and it will be a major issue in the future, that's a major issue in the future, that's a major issue in the future, that's a great warning. So, a simple example great warning. So, a simple example great warning. So, a simple example might be, hey, I noticed we're getting might be, hey, I noticed we're getting might be, hey, I noticed we're getting low on disc space. Okay, that's low on disc space. Okay, that's low on disc space. Okay, that's something that should be a warning something that should be a warning something that should be a warning because that's leading up to, hey, we're because that's leading up to, hey, we're because that's leading up to, hey, we're out of disc space. So, yes, that could out of disc space. So, yes, that could out of disc space. So, yes, that could be a good warning because again, you're be a good warning because again, you're be a good warning because again, you're going to do something about it. Maybe going to do something about it. Maybe going to do something about it. Maybe you won't do something about it right you won't do something about it right you won't do something about it right away, but you'll do something about it away, but you'll do something about it away, but you'll do something about it in a week or within the next few days as in a week or within the next few days as in a week or within the next few days as you get time because that warning let you get time because that warning let you get time because that warning let you know before it became a critical you know before it became a critical you know before it became a critical issue. So warnings are good to have, but issue. So warnings are good to have, but issue. So warnings are good to have, but again, don't log things as warnings that again, don't log things as warnings that again, don't log things as warnings that aren't precursors to actual errors or aren't precursors to actual errors or aren't precursors to actual errors or critical issues. If it's just a, hey, critical issues. If it's just a, hey, critical issues. If it's just a, hey, the user doesn't like us or, you know, the user doesn't like us or, you know, the user doesn't like us or, you know, hey, the user put bad data in, these are hey, the user put bad data in, these are hey, the user put bad data in, these are not warnings. This is just information

not warnings. This is just information not warnings. This is just information that you've elevated wrongly to a that you've elevated wrongly to a that you've elevated wrongly to a warning level. So again, it has to be warning level. So again, it has to be warning level. So again, it has to be based upon something that you're going based upon something that you're going based upon something that you're going to do something about. If you've noticed to do something about. If you've noticed to do something about. If you've noticed a theme so far, this is what logs are a theme so far, this is what logs are a theme so far, this is what logs are for. Logs are not for the packrats, the for. Logs are not for the packrats, the for. Logs are not for the packrats, the digital packrats who want to store all digital packrats who want to store all digital packrats who want to store all this information. That's not what logs this information. That's not what logs this information. That's not what logs are for. Logs or are for doing something are for. Logs or are for doing something are for. Logs or are for doing something with. If you're not doing something with with. If you're not doing something with with. If you're not doing something with your logs, you're not using them your logs, you're not using them your logs, you're not using them correctly. So, number four, understand correctly. So, number four, understand correctly. So, number four, understand your application's importance and adjust your application's importance and adjust your application's importance and adjust what you capture and where. what you capture and where. what you capture and where. What do we mean mean by this? Well, not What do we mean mean by this? Well, not What do we mean mean by this? Well, not all applications are created equal. Some all applications are created equal. Some all applications are created equal. Some applications are missionritical applications are missionritical applications are missionritical applications. I worked on a software as applications. I worked on a software as applications. I worked on a software as a service for a company that their whole a service for a company that their whole a service for a company that their whole line of business was a software as a line of business was a software as a line of business was a software as a service. If that software as a service service. If that software as a service service. If that software as a service went down, then we didn't get paid. All went down, then we didn't get paid. All went down, then we didn't get paid. All of our customers wouldn't pay us because of our customers wouldn't pay us because of our customers wouldn't pay us because we're not actually providing them what we're not actually providing them what we're not actually providing them what they need. The whole business would shut they need. The whole business would shut they need. The whole business would shut down. That's missionritical. And that down. That's missionritical. And that down. That's missionritical. And that application was logged differently. Now application was logged differently. Now application was logged differently. Now I've used or built other applications I've used or built other applications I've used or built other applications where it supported the development staff where it supported the development staff where it supported the development staff where it made deployments easier or it where it made deployments easier or it where it made deployments easier or it made you know clean up easier or made made you know clean up easier or made made you know clean up easier or made you know simple task documentation you know simple task documentation you know simple task documentation easier. These are not important easier. These are not important easier. These are not important applications. These are support applications. These are support applications. These are support applications that we could get by applications that we could get by applications that we could get by without or that are nice to have but no

without or that are nice to have but no without or that are nice to have but no one is coming in the weekend to fix. So one is coming in the weekend to fix. So one is coming in the weekend to fix. So you need to adjust your application's you need to adjust your application's you need to adjust your application's logs accordingly. logs accordingly. logs accordingly. So just because this application is a So just because this application is a So just because this application is a critical failure doesn't mean it's it's critical failure doesn't mean it's it's critical failure doesn't mean it's it's crazy important to to do. Again, you're crazy important to to do. Again, you're crazy important to to do. Again, you're not coming out of Sunday to fix this not coming out of Sunday to fix this not coming out of Sunday to fix this application if it's not a important application if it's not a important application if it's not a important application. So you need to understand application. So you need to understand application. So you need to understand your application's importance to the your application's importance to the your application's importance to the organization organization organization and adjust your logging accordingly. and adjust your logging accordingly. and adjust your logging accordingly. Also where you log because there's going Also where you log because there's going Also where you log because there's going to be the the central place that to be the the central place that to be the the central place that everybody should look at. Again, read everybody should look at. Again, read everybody should look at. Again, read your logs. Um everyone should look at your logs. Um everyone should look at your logs. Um everyone should look at that tells you what's going on, what's that tells you what's going on, what's that tells you what's going on, what's critical, what's an error, what's a critical, what's an error, what's a critical, what's an error, what's a warning. Those are things you should be warning. Those are things you should be warning. Those are things you should be looking at probably on a daily basis. looking at probably on a daily basis. looking at probably on a daily basis. You should be aware of what's going on You should be aware of what's going on You should be aware of what's going on and be fixing those things because and be fixing those things because and be fixing those things because they're priorities. But if you have they're priorities. But if you have they're priorities. But if you have minor applications, they probably minor applications, they probably minor applications, they probably shouldn't go in the same system. Maybe shouldn't go in the same system. Maybe shouldn't go in the same system. Maybe you should be logged to a text file or you should be logged to a text file or you should be logged to a text file or maybe you should log to a secondary maybe you should log to a secondary maybe you should log to a secondary server that you check once in a while.

server that you check once in a while. server that you check once in a while. When you have time, when you're looking When you have time, when you're looking When you have time, when you're looking to um make a change to a secondary to um make a change to a secondary to um make a change to a secondary system, you also look and see, hey, are system, you also look and see, hey, are system, you also look and see, hey, are there any critical or error warnings in there any critical or error warnings in there any critical or error warnings in there that I should address as well? So there that I should address as well? So there that I should address as well? So where you log is also important because where you log is also important because where you log is also important because also logging is expensive and you don't also logging is expensive and you don't also logging is expensive and you don't need to be logging lots of messages from need to be logging lots of messages from need to be logging lots of messages from non-critical systems into your main non-critical systems into your main non-critical systems into your main logging system. That's going to cost logging system. That's going to cost logging system. That's going to cost more money and again it's going to clog more money and again it's going to clog more money and again it's going to clog up that main system with a whole bunch up that main system with a whole bunch up that main system with a whole bunch of critical error and warnings that of critical error and warnings that of critical error and warnings that aren't actually critical errors or aren't actually critical errors or aren't actually critical errors or warnings that need to be dealt with warnings that need to be dealt with warnings that need to be dealt with right away. So, make sure not to clog up right away. So, make sure not to clog up right away. So, make sure not to clog up your systems with too many false your systems with too many false your systems with too many false positives. Try very hard to have zero positives. Try very hard to have zero positives. Try very hard to have zero false positives. false positives. false positives. Number six, do not capture information Number six, do not capture information Number six, do not capture information you're not going to act on every day you're not going to act on every day you're not going to act on every day until unless you're investigating an until unless you're investigating an until unless you're investigating an issue. Okay? So, when you're capturing issue. Okay? So, when you're capturing issue. Okay? So, when you're capturing information, don't don't gather information, don't don't gather information, don't don't gather information you're not going to act on.

information you're not going to act on. information you're not going to act on. So if if you're saying well I you know I So if if you're saying well I you know I So if if you're saying well I you know I need to know what the user does when need to know what the user does when need to know what the user does when they use the application do you or is they use the application do you or is they use the application do you or is that more like a testing thing or is that more like a testing thing or is that more like a testing thing or is that more like a a sometimes thing and that more like a a sometimes thing and that more like a a sometimes thing and that's where you have things like that's where you have things like that's where you have things like information logs where you can turn information logs where you can turn information logs where you can turn those on just for investigating an issue those on just for investigating an issue those on just for investigating an issue or just for um you know trying to figure or just for um you know trying to figure or just for um you know trying to figure out how users are using your out how users are using your out how users are using your application. Not all the time. So don't application. Not all the time. So don't application. Not all the time. So don't log information. Don't capture things log information. Don't capture things log information. Don't capture things that you aren't going to act on. That that you aren't going to act on. That that you aren't going to act on. That also means what types of data you also means what types of data you also means what types of data you capture. So this might be, you know capture. So this might be, you know capture. So this might be, you know what, we need to know that we're out of what, we need to know that we're out of what, we need to know that we're out of dispace, but we don't need to know what dispace, but we don't need to know what dispace, but we don't need to know what you were doing when you found out you're you were doing when you found out you're you were doing when you found out you're out of dispace. We don't need to know out of dispace. We don't need to know out of dispace. We don't need to know that information. We just need to know that information. We just need to know that information. We just need to know that the system went, "Hey, I tried to that the system went, "Hey, I tried to that the system went, "Hey, I tried to save a a file and we're out of dispace." save a a file and we're out of dispace." save a a file and we're out of dispace." That's all you need to know. You don't That's all you need to know. You don't That's all you need to know. You don't need to know who was who was trying to need to know who was who was trying to need to know who was who was trying to do something. it. That's irrelevant do something. it. That's irrelevant do something. it. That's irrelevant information. Capture the information information. Capture the information information. Capture the information you're actually going to act on. And you're actually going to act on. And you're actually going to act on. And again, you can use those lower log again, you can use those lower log again, you can use those lower log levels for when you're investigating levels for when you're investigating levels for when you're investigating issues or investigating how the system issues or investigating how the system issues or investigating how the system is used, etc. You can turn those on, is used, etc. You can turn those on, is used, etc. You can turn those on, maybe log them somewhere else or maybe maybe log them somewhere else or maybe maybe log them somewhere else or maybe just make sure they're filtered out of just make sure they're filtered out of just make sure they're filtered out of the main system and only used for for the main system and only used for for the main system and only used for for those investigations.

those investigations. those investigations. Number seven, do not save data beyond Number seven, do not save data beyond Number seven, do not save data beyond when you will reasonably use it. I have when you will reasonably use it. I have when you will reasonably use it. I have found that in most cases, 30 days is a found that in most cases, 30 days is a found that in most cases, 30 days is a is about all you need because if you get is about all you need because if you get is about all you need because if you get beyond that, you're not actually using beyond that, you're not actually using beyond that, you're not actually using the data. First of all, if you have a the data. First of all, if you have a the data. First of all, if you have a critical issue that stays in the critical issue that stays in the critical issue that stays in the critical issues list for over 30 days, critical issues list for over 30 days, critical issues list for over 30 days, you're not using the critical data you're not using the critical data you're not using the critical data system correctly. You need to have system correctly. You need to have system correctly. You need to have addressed it before now. that needs to addressed it before now. that needs to addressed it before now. that needs to be turned into a ticket that needs to be be turned into a ticket that needs to be be turned into a ticket that needs to be starting worked on right away. So if you starting worked on right away. So if you starting worked on right away. So if you have data for more than 30 days, why why have data for more than 30 days, why why have data for more than 30 days, why why is it there? Why is it not converted is it there? Why is it not converted is it there? Why is it not converted into an issue? For the most part, delete into an issue? For the most part, delete into an issue? For the most part, delete it after 30 days. That's all you need at it after 30 days. That's all you need at it after 30 days. That's all you need at most. So you can have a log system that most. So you can have a log system that most. So you can have a log system that turns over quite frequently. That even turns over quite frequently. That even turns over quite frequently. That even handles things like, hey, we had handles things like, hey, we had handles things like, hey, we had Christmas break and we're off for a Christmas break and we're off for a Christmas break and we're off for a week. No problem. Because you have 30 week. No problem. Because you have 30 week. No problem. Because you have 30 days. you still have 21 days of usable days. you still have 21 days of usable days. you still have 21 days of usable data. So there's still a lot of data data. So there's still a lot of data data. So there's still a lot of data there for you. Even if you have breaks, there for you. Even if you have breaks, there for you. Even if you have breaks, even if you have people go on vacation, even if you have people go on vacation, even if you have people go on vacation, even if you have both combined at the even if you have both combined at the even if you have both combined at the same time, like 30 days is a lot of same time, like 30 days is a lot of same time, like 30 days is a lot of data. If you are acting on it, the times data. If you are acting on it, the times data. If you are acting on it, the times when I've seen people say, I need more when I've seen people say, I need more when I've seen people say, I need more than 30 days, is when they are not than 30 days, is when they are not than 30 days, is when they are not acting on it and they are only looking acting on it and they are only looking acting on it and they are only looking at it every once in a while. They say, at it every once in a while. They say, at it every once in a while. They say, "Well, I need to know historically how "Well, I need to know historically how "Well, I need to know historically how things are working." You should have things are working." You should have things are working." You should have been fixing these problems. So, been fixing these problems. So, been fixing these problems. So, historically shouldn't matter because

historically shouldn't matter because historically shouldn't matter because you should have fixed these problems by you should have fixed these problems by you should have fixed these problems by now. So, therefore, if you have a now. So, therefore, if you have a now. So, therefore, if you have a critical error or critical issue that's critical error or critical issue that's critical error or critical issue that's happened over 30 days long, then you're happened over 30 days long, then you're happened over 30 days long, then you're doing something wrong or it's it's in a doing something wrong or it's it's in a doing something wrong or it's it's in a ticket. You're trying to fix it, but at ticket. You're trying to fix it, but at ticket. You're trying to fix it, but at that point, it's already on ticket. So, that point, it's already on ticket. So, that point, it's already on ticket. So, why you need it in a logging system? So why you need it in a logging system? So why you need it in a logging system? So try not to save data beyond when it's try not to save data beyond when it's try not to save data beyond when it's reasonably useful. Otherwise, what reasonably useful. Otherwise, what reasonably useful. Otherwise, what happens is you become a digital packrat happens is you become a digital packrat happens is you become a digital packrat and you got so much information there and you got so much information there and you got so much information there that you just never use it, but just that you just never use it, but just that you just never use it, but just keeps getting more and more expensive to keeps getting more and more expensive to keeps getting more and more expensive to store and it can actually cause some store and it can actually cause some store and it can actually cause some other problems where you are storing other problems where you are storing other problems where you are storing things beyond when you should. things beyond when you should. things beyond when you should. So don't save data beyond when it's So don't save data beyond when it's So don't save data beyond when it's reasonably useful. Number eight, do not reasonably useful. Number eight, do not reasonably useful. Number eight, do not log sensitive data unless it's log sensitive data unless it's log sensitive data unless it's absolutely necessary. absolutely necessary. absolutely necessary. This is one that often trips people up This is one that often trips people up This is one that often trips people up and this is where a lot of security and this is where a lot of security and this is where a lot of security breaches can come in because you're breaches can come in because you're breaches can come in because you're logging the sensitive data and taking it logging the sensitive data and taking it logging the sensitive data and taking it out of the secure system and putting it out of the secure system and putting it out of the secure system and putting it into a less secure system. So, first of into a less secure system. So, first of into a less secure system. So, first of all, your logging system should be all, your logging system should be all, your logging system should be pretty secure. And then you should also pretty secure. And then you should also pretty secure. And then you should also try not to include anything that's of try not to include anything that's of try not to include anything that's of person identifiable information or other person identifiable information or other person identifiable information or other sensitive data in your logs.

sensitive data in your logs. sensitive data in your logs. Yes, there's exceptions. Yes, there are Yes, there's exceptions. Yes, there are Yes, there's exceptions. Yes, there are times when you might have to. But in times when you might have to. But in times when you might have to. But in that case, you need to make sure that that case, you need to make sure that that case, you need to make sure that your logging system is locked down your logging system is locked down your logging system is locked down secure and it's also in compliance with secure and it's also in compliance with secure and it's also in compliance with all the regulations around forgetting a all the regulations around forgetting a all the regulations around forgetting a user if they ask you to. That means you user if they ask you to. That means you user if they ask you to. That means you need to go through your log system and need to go through your log system and need to go through your log system and clean out those logs. referring to that clean out those logs. referring to that clean out those logs. referring to that user or change that data. So user or change that data. So user or change that data. So being compliance is harder if you're being compliance is harder if you're being compliance is harder if you're logging sensitive information. So maybe logging sensitive information. So maybe logging sensitive information. So maybe try not to. And again, these regulations try not to. And again, these regulations try not to. And again, these regulations are there for a reason because people are there for a reason because people are there for a reason because people are putting sensitive data in systems are putting sensitive data in systems are putting sensitive data in systems that aren't as secure. They think, well, that aren't as secure. They think, well, that aren't as secure. They think, well, it's just a logging system or it's not it's just a logging system or it's not it's just a logging system or it's not as as protected because, well, everyone as as protected because, well, everyone as as protected because, well, everyone has to have access to it, right? and has to have access to it, right? and has to have access to it, right? and then next thing you know you're leaking then next thing you know you're leaking then next thing you know you're leaking sensitive data and and endangering your sensitive data and and endangering your sensitive data and and endangering your users because you were sloppy. Don't do users because you were sloppy. Don't do users because you were sloppy. Don't do that. So number nine, that. So number nine, that. So number nine, number nine is don't log to the number nine is don't log to the number nine is don't log to the browser's console in production.

browser's console in production. browser's console in production. This is one where you know if you're This is one where you know if you're This is one where you know if you're using something like uh react or angular using something like uh react or angular using something like uh react or angular or view uh or even like blaze web or view uh or even like blaze web or view uh or even like blaze web assembly where you are are front end assembly where you are are front end assembly where you are are front end fully fully fully try really hard not to put console log try really hard not to put console log try really hard not to put console log statements there. It use an actual statements there. It use an actual statements there. It use an actual logging service. Now sometimes that logging service. Now sometimes that logging service. Now sometimes that logging service is not available where logging service is not available where logging service is not available where maybe you have a network issue or other maybe you have a network issue or other maybe you have a network issue or other thing where you need to log um or tell a thing where you need to log um or tell a thing where you need to log um or tell a user in some way. Try and tell the user user in some way. Try and tell the user user in some way. Try and tell the user directly. Don't try very hard not to put directly. Don't try very hard not to put directly. Don't try very hard not to put it into the console log. We really don't it into the console log. We really don't it into the console log. We really don't want to have to ask users to open up the want to have to ask users to open up the want to have to ask users to open up the developer console, look at the actual developer console, look at the actual developer console, look at the actual console message, and figure out what the console message, and figure out what the console message, and figure out what the message says. That's not good. So, if message says. That's not good. So, if message says. That's not good. So, if we're not asking them to do that, then we're not asking them to do that, then we're not asking them to do that, then why are we putting those messages in why are we putting those messages in why are we putting those messages in there? Because the other thing is those there? Because the other thing is those there? Because the other thing is those messages can leak information about messages can leak information about messages can leak information about what's going on. Now everything you do what's going on. Now everything you do what's going on. Now everything you do in the browser is going to be unsecured in the browser is going to be unsecured in the browser is going to be unsecured or at least unsecured as as for that or at least unsecured as as for that or at least unsecured as as for that person. So if that person logged in, person. So if that person logged in, person. So if that person logged in, they have their credentials. Well, sure they have their credentials. Well, sure they have their credentials. Well, sure they have access to all their things.

they have access to all their things. they have access to all their things. But that also means that any code that But that also means that any code that But that also means that any code that runs in the front end is going to be runs in the front end is going to be runs in the front end is going to be completely exposed to the user. They can completely exposed to the user. They can completely exposed to the user. They can read all the code. They can read all the read all the code. They can read all the read all the code. They can read all the data. So, it's not quite the same as data. So, it's not quite the same as data. So, it's not quite the same as logging like a a C applications logging like a a C applications logging like a a C applications information and and displaying that. You information and and displaying that. You information and and displaying that. You know, if you have a Blazer uh web app know, if you have a Blazer uh web app know, if you have a Blazer uh web app where you're showing the server side where you're showing the server side where you're showing the server side logs, yeah, it's more that's more logs, yeah, it's more that's more logs, yeah, it's more that's more information leakage than a console log information leakage than a console log information leakage than a console log would be, but still don't put it in the would be, but still don't put it in the would be, but still don't put it in the console logs. Not if you help it. There console logs. Not if you help it. There console logs. Not if you help it. There are exceptions. There are exception, you are exceptions. There are exception, you are exceptions. There are exception, you know, no no pun intended, but there are know, no no pun intended, but there are know, no no pun intended, but there are exceptions to that rule. But for the exceptions to that rule. But for the exceptions to that rule. But for the most part, try not to put um errors in most part, try not to put um errors in most part, try not to put um errors in your console. Try and expose it to the your console. Try and expose it to the your console. Try and expose it to the user or send it to a logging system if user or send it to a logging system if user or send it to a logging system if you're going to act on it. If your logs you're going to act on it. If your logs you're going to act on it. If your logs have too much information, they become have too much information, they become have too much information, they become useless. If your application treats too useless. If your application treats too useless. If your application treats too many things as important, nothing is many things as important, nothing is many things as important, nothing is important. And if you aren't addressing important. And if you aren't addressing important. And if you aren't addressing the actual issues as they come up, the actual issues as they come up, the actual issues as they come up, you're going to hide the real issues.

you're going to hide the real issues. you're going to hide the real issues. You need to make sure you're only You need to make sure you're only You need to make sure you're only looking at logs you'll actually act looking at logs you'll actually act looking at logs you'll actually act upon. That's how you protect your upon. That's how you protect your upon. That's how you protect your applications. It's how you improve your applications. It's how you improve your applications. It's how you improve your user experience and it's how you ensure user experience and it's how you ensure user experience and it's how you ensure you'll actually get value out of your you'll actually get value out of your you'll actually get value out of your logs. Thanks for listening. As always, I logs. Thanks for listening. As always, I logs. Thanks for listening. As always, I am Tim Corey.