I am a problem solver. When I see a problem that I can fix I fix it. Which makes it really frustrating for me when people won't let me solve their problem.
Here is an example I hit in database design all the time. People want their next version of software to have all these amazing new features but they don't want to change the database design to properly support these amazing new features. Sometimes you can support a new feature by just adding a few new tables to a database, other times you have to redesign the entire database because the feature you are adding is so fundamental. But that takes time and adds risk to a project so they want you to support the new feature without redesigning the entire database, even if that is impossible.
A non-database example is when you give the people the solution to the problem but they reject it because they don't understand it or the solution is too different from what they think the solution should be. This is frustrating because I have given them the answer and I know it is right, but the problem continues because I can't convince them. If they would just try my answer they would see that it works, but they won't even do that.
To a problem solver the only thing more frustrating than a problem you can't solve is a problem you have a solution to but you aren't allowed to implement the solution.
Wednesday, October 21, 2009
Tuesday, September 29, 2009
Database Design Insights
As I mentioned before I have been spending some time rethinking some of the designs I have done or have seen over my career. I was able to come up with a few insights.
First, not enough companies keep history of who changed records, when, and what the changes were. Keeping this sort of information is critical to reconstructing why a decision was made. You need to be able to tell the state of the database at a given date and time to know why a decision was made the way it was. Another bit of information that is needed for sensitive data is who viewed the data and when. None of the companies I have worked for have ever kept a log of select statements.
Another insight was that most tables fall into basic categories. I usually hate putting things in boxes but had to create these as a way to explain table design to developers. Reference tables contain information that is fairly basic such as units of measure or model years. These tables are rarely updated but often read. Cross reference tables are used to translate one piece of information into another (employee 276 in the old HR system is employee 889 in the new HR system). These tables are again rarely updated and are less commonly read. Master tables contain information about customers, parts, suppliers, etc. These tables are updated fairly often and read fairly often. Transaction tables contain information about invoices, orders, and other event driven processes. These tables are added to constantly and often read.
I spent a lot of time thinking about dates. Dates are often represented with times attached, which is fine if you know precisely when something happened. The aren't as convenient when you want to record a birth that happened in the spring of 1847 or an political movement that occured in the later half of the 1500's. I have found a few ways to represent non-exact dates but they are alien to the database systems we have and therefore not a clean solution. I need to do some more work on this.
While thinking about imprecise dates I considered imprecise data. Suppose you have multiple reports of when an event happened or multiple descriptions of a bank robber? Every system I have dealt with expects one precise record, but that isn't how the real world works. I was able to come up with a system that allows for multiple versions of an event or fact, including a reliability factor. But I need to work on this more and haven't decided how multiple versions of a fact can be accurately reflected on a report.
So, nothing earth-shattering. No cures for cancer. And perhaps I am the only one who cares about such things. Still, here are my thoughts.
First, not enough companies keep history of who changed records, when, and what the changes were. Keeping this sort of information is critical to reconstructing why a decision was made. You need to be able to tell the state of the database at a given date and time to know why a decision was made the way it was. Another bit of information that is needed for sensitive data is who viewed the data and when. None of the companies I have worked for have ever kept a log of select statements.
Another insight was that most tables fall into basic categories. I usually hate putting things in boxes but had to create these as a way to explain table design to developers. Reference tables contain information that is fairly basic such as units of measure or model years. These tables are rarely updated but often read. Cross reference tables are used to translate one piece of information into another (employee 276 in the old HR system is employee 889 in the new HR system). These tables are again rarely updated and are less commonly read. Master tables contain information about customers, parts, suppliers, etc. These tables are updated fairly often and read fairly often. Transaction tables contain information about invoices, orders, and other event driven processes. These tables are added to constantly and often read.
I spent a lot of time thinking about dates. Dates are often represented with times attached, which is fine if you know precisely when something happened. The aren't as convenient when you want to record a birth that happened in the spring of 1847 or an political movement that occured in the later half of the 1500's. I have found a few ways to represent non-exact dates but they are alien to the database systems we have and therefore not a clean solution. I need to do some more work on this.
While thinking about imprecise dates I considered imprecise data. Suppose you have multiple reports of when an event happened or multiple descriptions of a bank robber? Every system I have dealt with expects one precise record, but that isn't how the real world works. I was able to come up with a system that allows for multiple versions of an event or fact, including a reliability factor. But I need to work on this more and haven't decided how multiple versions of a fact can be accurately reflected on a report.
So, nothing earth-shattering. No cures for cancer. And perhaps I am the only one who cares about such things. Still, here are my thoughts.
Wednesday, September 23, 2009
Life and stuff
I haven't blogged in quite a while. It has been a rough summer.
I joined a Tai Chi class which I really liked, but had to drop out because of pain. My legs just can't handle the bent knee posture required. Perhaps in another 6 months or a year I will try again, after I have built up my knee and leg strength.
Work has been rough, fortunately I have been able to stay employed although we did have to do a few weeks of furlough. I am hopeful that the worst is over and I can continue to stay employed, but we had to lose a lot of good people and the workload is going to be tough once they start producing cars again.
The only groundbreaking thing I have achieved all summer was in a dream. For the first time in 50 years I saw the sky in a dream. Up until now all of my dreams have been indoors or in heavily wooded areas, but once this summer I dreamed of being outside and actually looked up and saw the sky. Not an earthshaking accomplishment, but it has cleared up an odd omission that has been going on for 50 years.
I have accomplished a few things this summer. Minor things like cleaning up the garage and getting some house repairs done. But even minor things are better than nothing.
I have also done some cool things at work, but can't discuss them due to confidentiality issues. The work led in some interesting directions and gave me some new insights into database design. I have some hopes that I can write about some of the insights in the future.
I joined a Tai Chi class which I really liked, but had to drop out because of pain. My legs just can't handle the bent knee posture required. Perhaps in another 6 months or a year I will try again, after I have built up my knee and leg strength.
Work has been rough, fortunately I have been able to stay employed although we did have to do a few weeks of furlough. I am hopeful that the worst is over and I can continue to stay employed, but we had to lose a lot of good people and the workload is going to be tough once they start producing cars again.
The only groundbreaking thing I have achieved all summer was in a dream. For the first time in 50 years I saw the sky in a dream. Up until now all of my dreams have been indoors or in heavily wooded areas, but once this summer I dreamed of being outside and actually looked up and saw the sky. Not an earthshaking accomplishment, but it has cleared up an odd omission that has been going on for 50 years.
I have accomplished a few things this summer. Minor things like cleaning up the garage and getting some house repairs done. But even minor things are better than nothing.
I have also done some cool things at work, but can't discuss them due to confidentiality issues. The work led in some interesting directions and gave me some new insights into database design. I have some hopes that I can write about some of the insights in the future.
Friday, August 14, 2009
Some companies seem a bit suicidal
I have noticed a trend lately. Many of the companies I deal with seem to be deliberately trying to annoy their customers. In these difficult economic times I really don't understand this.
I used to do most of my shopping at Meijer. Then they started reducing the number of cashiers so that long lines would form at the register. That was followed by replacing most of the cashiers with self-checkout lanes. While some people like them I hate self-checkout lanes, and so do quite a few others judging by the long lines of people waiting to check out at the few lanes that still have cashiers. The result is that while I used to spend over $100 a week at Meijer I now spend less than $20, and would stop spending that much if I could find another brand of chocolate milk my son likes.
I used to shop at Home Depot a lot, but now you have to walk through a gauntlet of no less than four people trying to sell you roof repairs, some special product, a credit card, or something similar. Now I do most of my shopping at Lowes.
I rarely sign up for email of special offers from companies, but for a few of my favorites I have. Of these I have had to drop out of two thirds of them because they started sending me daily emails. Do they really think they can harass me into buying something?
I realize all of these are attempts to save money and increase sales, but you don't accomplish those things by driving away your customers. The people that run these companies surely must know that, they are customers themselves. Yet the companies do it anyway. Either you can truly harass your customers into buying more or these companies have become so desperate they are suicidal. I am not a retail genius but I am leaning towards the second explanation.
I used to do most of my shopping at Meijer. Then they started reducing the number of cashiers so that long lines would form at the register. That was followed by replacing most of the cashiers with self-checkout lanes. While some people like them I hate self-checkout lanes, and so do quite a few others judging by the long lines of people waiting to check out at the few lanes that still have cashiers. The result is that while I used to spend over $100 a week at Meijer I now spend less than $20, and would stop spending that much if I could find another brand of chocolate milk my son likes.
I used to shop at Home Depot a lot, but now you have to walk through a gauntlet of no less than four people trying to sell you roof repairs, some special product, a credit card, or something similar. Now I do most of my shopping at Lowes.
I rarely sign up for email of special offers from companies, but for a few of my favorites I have. Of these I have had to drop out of two thirds of them because they started sending me daily emails. Do they really think they can harass me into buying something?
I realize all of these are attempts to save money and increase sales, but you don't accomplish those things by driving away your customers. The people that run these companies surely must know that, they are customers themselves. Yet the companies do it anyway. Either you can truly harass your customers into buying more or these companies have become so desperate they are suicidal. I am not a retail genius but I am leaning towards the second explanation.
Monday, August 3, 2009
Symmetry
I haven't posted in a long time, I was working a lot of hours on a project at work. But now I am starting to recover so time to post something.
A while ago I told somebody that a particular situation offended my sense of symmetry. It started me thinking and I realized that a lot of the way I think has to do with symmetry. My concept of fairness reflects symmetry in that I believe each side of a deal should be equal. My database designs usually reflect symmetry in the way I design the relationships between tables.
Sometimes however you don't have symmetry. A tree growing naturally is rarely symmetrical, yet it is not any less beautiful. In real life you can't always have symmetry, and certain situations it would be bad if you did (parents should punish a child but a child should not be able to punish his parents).
My need for symmetry puts me at odds with the real world where you can't always get symmetry and sometimes don't want it. Yet my sense of symmetry has served me well in my work and some of my real life dealings.
In a practical sense my need for symmetry is both good and bad. It has allowed me to create great database designs yet has caused me serious frustration in dealing with the real world. How do you deal with this?
A while ago I told somebody that a particular situation offended my sense of symmetry. It started me thinking and I realized that a lot of the way I think has to do with symmetry. My concept of fairness reflects symmetry in that I believe each side of a deal should be equal. My database designs usually reflect symmetry in the way I design the relationships between tables.
Sometimes however you don't have symmetry. A tree growing naturally is rarely symmetrical, yet it is not any less beautiful. In real life you can't always have symmetry, and certain situations it would be bad if you did (parents should punish a child but a child should not be able to punish his parents).
My need for symmetry puts me at odds with the real world where you can't always get symmetry and sometimes don't want it. Yet my sense of symmetry has served me well in my work and some of my real life dealings.
In a practical sense my need for symmetry is both good and bad. It has allowed me to create great database designs yet has caused me serious frustration in dealing with the real world. How do you deal with this?
Tuesday, May 12, 2009
The majority are always sane - Ringworld by Larry Niven
When I first started college I studied Psychology. After taking a couple of classes I figured out that the so called science of psychology was a bunch of garbage. So I got into computers instead.
In the book Ringworld a character states "The majority are always sane". If you define insanity as being different than the norm then this is true. If you define insanity as being irrational then this is not true.
The problem I had with psychology was that it tried to define sanity as being "normal". Behavior that was different than normal was something that needed to be treated. I felt this was wrong, that sanity had more to do with rationality than what the majority of people do or think. Over the years I have seen plenty of evidence that I was right, and yet the majority of the population (and I am sure the majority of psychologists) still believe sanity is about being normal.
So let me put out some arguments to explain my reasoning.
The vast majority of people have never killed anyone. Killing people is considered bad and certainly isn't normal. Does this mean policemen and soldiers are insane? If you define sanity as being normal then they must be insane. If you define sanity as rationality then they are sane because they are killing people for a reason, usually a well thought out reason that caused them to accept the position they are in.
Long ago the vast majority of people thought the world was flat. Does that mean that anyone that thought the world was round was insane? People at the time certainly thought so. But the people who thought the world was round had a well thought out reason why the world was round. They were rational, but not normal. It also turns out they were right.
To summarize: Sanity can't be determined by what the majority believe. Being outside of the norm doesn't mean a person is insane. On the flip side, being rational isn't a guarantee of sanity. A person can be rational for most of their life then suddenly slip into insanity. Or they may be rational on some subjects and irrational on others.
To summarize my summary: People are complicated. Learn to live with it.
In the book Ringworld a character states "The majority are always sane". If you define insanity as being different than the norm then this is true. If you define insanity as being irrational then this is not true.
The problem I had with psychology was that it tried to define sanity as being "normal". Behavior that was different than normal was something that needed to be treated. I felt this was wrong, that sanity had more to do with rationality than what the majority of people do or think. Over the years I have seen plenty of evidence that I was right, and yet the majority of the population (and I am sure the majority of psychologists) still believe sanity is about being normal.
So let me put out some arguments to explain my reasoning.
The vast majority of people have never killed anyone. Killing people is considered bad and certainly isn't normal. Does this mean policemen and soldiers are insane? If you define sanity as being normal then they must be insane. If you define sanity as rationality then they are sane because they are killing people for a reason, usually a well thought out reason that caused them to accept the position they are in.
Long ago the vast majority of people thought the world was flat. Does that mean that anyone that thought the world was round was insane? People at the time certainly thought so. But the people who thought the world was round had a well thought out reason why the world was round. They were rational, but not normal. It also turns out they were right.
To summarize: Sanity can't be determined by what the majority believe. Being outside of the norm doesn't mean a person is insane. On the flip side, being rational isn't a guarantee of sanity. A person can be rational for most of their life then suddenly slip into insanity. Or they may be rational on some subjects and irrational on others.
To summarize my summary: People are complicated. Learn to live with it.
Monday, May 11, 2009
Argh
I was born a natural problem solver. It is the one ability I have true excellence in. I am happiest when I am given a huge mess and asked to clean it up. The reason I got into computers in the first place was because it was all problem solving.
But right now with Michigan's economy being in the dumper businesses are dealing with problems far beyond my ability to solve like what if our largest customer goes bankrupt, and they don't have the time to deal with the issues that I could possibly solve like giving them a computerized knowledgebase or redesigning their accounting system. My current place of employment has cut IT staff to such low levels we are having a hard time supporting the computer systems we have in place now, so even if they wanted an innovative solution we don't have the staff to deliver it.
What does a natural problem solver do when all the problems are beyond his ability to solve? He sits and gets very frustrated. Lately I have been exploring the very edges of theoretical database design, object/relational databases, design theory, etc. But it isn't the same. I need a problem to solve. Of course due to work agreements and conflict of interest issues that problem would have to come from my employer and that isn't going to be happening soon.
It is all very frustrating.
But right now with Michigan's economy being in the dumper businesses are dealing with problems far beyond my ability to solve like what if our largest customer goes bankrupt, and they don't have the time to deal with the issues that I could possibly solve like giving them a computerized knowledgebase or redesigning their accounting system. My current place of employment has cut IT staff to such low levels we are having a hard time supporting the computer systems we have in place now, so even if they wanted an innovative solution we don't have the staff to deliver it.
What does a natural problem solver do when all the problems are beyond his ability to solve? He sits and gets very frustrated. Lately I have been exploring the very edges of theoretical database design, object/relational databases, design theory, etc. But it isn't the same. I need a problem to solve. Of course due to work agreements and conflict of interest issues that problem would have to come from my employer and that isn't going to be happening soon.
It is all very frustrating.
Subscribe to:
Posts (Atom)