Thread: Thoughts on "Love Your Database"
<snip>
The sensible way is to do it as John wrote - to restrict access rights
to everyone except admin to calling functions only. That way the
functions are written by the people who are paid to understand the
business rules and the data behind it, and the application developers
can ask those experts to do the heavy lifting for them. Having to
persuade management that they should no longer be able to connect the
database to MS Access and make changes that way will usually put an
end to that pure model, though. :)Allowing PHBs direct access to company data is a nasty thing. They become like some users who "know Excel". They are now just as knowledgeable as someone who's been doing this for years. I've actually heard one say something akin to: "Damn it, I can write Excel formulas. I know very well that an new function on the web site could be written in less than a day, if you'd just get off you a$$ and do it."
Geoff--The unfacts, did we have them, are too imprecisely few to warrant our certitude.Maranatha! <><
John McKown
from the INFORMATION_SCHEMA would be a good start.
--
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.
The reality is somewhere in the middle as it often is. My point is you don't have to replace a million lines of code if you plan upfront. If you don't .. you do.
On Wed, May 04, 2016 11:05:25 AM Will McCormick wrote:
A reason to consider may be portability. What happens if I want to let my customer chose their data store or I just don't want to put all my eggs in one basket.Technically there are truths but you cannot ignore the business side either. If a we can exceed our performance requirements and keep things generic/portable this is the best of both worlds.I think this is the main reason people separate the business logic from the database. How many of you have ported databases between platforms? Or had multiple types of data stores in the same company?
I have been waiting for the portability argument for the last 20+ posts :-) Everyone who did any type of consulting/working in this field knows that latest the second argument from management is “portability”. I have yet to see anyone who really needed to move to a different database. If they did it usually is a political issue and not a technical one (uhh, we hired this new operations manager and MS SQL is so much better than postgresql …) Unless you develop a generic software to be sold to many clients, the choice of data storage is rarely a real concern unless someone who feels the urge to be important starts throwing words he read in a magazine at management. None of my clients ever questioned the database. They either had one which they wanted to use (i.e. they had a big iron IBM with DB2 on it plus a DBA who knew what he was doing) or they didn't care as long as it worked as expected and came with a maintenance contract of sorts.
If you're developing a web application, the least concern is portability (you're not going to replace the million lines of PHP either). The biggest concern should lie with data integrity and API's that protect your data and business logic so you can hire that cheap app development company from overseas your boss recommended (based on a spam email) without running the risk of compromising everything.
Uwe
I agree it's typically political but so are most things business. Examples: Companies buy other companies - You are using a competitors data store and want to replace it. Company needs to compete with competitors and wants to reduce cost ... these are not technical requirements and it's certainly not vapor ideology. I have only worked for startups and have seen this happen at every company i have worked for, yes it is political but yes it happens. Smaller companies are more susceptible to it.
The reality is somewhere in the middle as it often is. My point is you don't have to replace a million lines of code if you plan upfront. If you don't .. you do.On Wed, May 4, 2016 at 11:29 AM, Uwe Schroeder <uwe@oss4u.com> wrote:On Wed, May 04, 2016 11:05:25 AM Will McCormick wrote:
A reason to consider may be portability. What happens if I want to let my customer chose their data store or I just don't want to put all my eggs in one basket.Technically there are truths but you cannot ignore the business side either. If a we can exceed our performance requirements and keep things generic/portable this is the best of both worlds.I think this is the main reason people separate the business logic from the database. How many of you have ported databases between platforms? Or had multiple types of data stores in the same company?
I have been waiting for the portability argument for the last 20+ posts :-) Everyone who did any type of consulting/working in this field knows that latest the second argument from management is “portability”. I have yet to see anyone who really needed to move to a different database. If they did it usually is a political issue and not a technical one (uhh, we hired this new operations manager and MS SQL is so much better than postgresql …) Unless you develop a generic software to be sold to many clients, the choice of data storage is rarely a real concern unless someone who feels the urge to be important starts throwing words he read in a magazine at management. None of my clients ever questioned the database. They either had one which they wanted to use (i.e. they had a big iron IBM with DB2 on it plus a DBA who knew what he was doing) or they didn't care as long as it worked as expected and came with a maintenance contract of sorts.
If you're developing a web application, the least concern is portability (you're not going to replace the million lines of PHP either). The biggest concern should lie with data integrity and API's that protect your data and business logic so you can hire that cheap app development company from overseas your boss recommended (based on a spam email) without running the risk of compromising everything.
Uwe
I'm not sure that when a company buys another company they can just migrate the other database without any logic layer. The data is usually useless without the business layer which tells how to merge all the parts together to have a simple answer to a question like "do we have this in stock". And for such a migration that's not too important if we have the logic in database, or in some other layer. Of course it is always simpler to migrate a database treated like a CSV file, where all the logic (including constraints) is in an external application. But do we really want that?On 4 May 2016 at 18:14, Will McCormick <wmccormick@gmail.com> wrote:I agree it's typically political but so are most things business. Examples: Companies buy other companies - You are using a competitors data store and want to replace it. Company needs to compete with competitors and wants to reduce cost ... these are not technical requirements and it's certainly not vapor ideology. I have only worked for startups and have seen this happen at every company i have worked for, yes it is political but yes it happens. Smaller companies are more susceptible to it.
The reality is somewhere in the middle as it often is. My point is you don't have to replace a million lines of code if you plan upfront. If you don't .. you do.On Wed, May 4, 2016 at 11:29 AM, Uwe Schroeder <uwe@oss4u.com> wrote:On Wed, May 04, 2016 11:05:25 AM Will McCormick wrote:
A reason to consider may be portability. What happens if I want to let my customer chose their data store or I just don't want to put all my eggs in one basket.Technically there are truths but you cannot ignore the business side either. If a we can exceed our performance requirements and keep things generic/portable this is the best of both worlds.I think this is the main reason people separate the business logic from the database. How many of you have ported databases between platforms? Or had multiple types of data stores in the same company?
I have been waiting for the portability argument for the last 20+ posts :-) Everyone who did any type of consulting/working in this field knows that latest the second argument from management is “portability”. I have yet to see anyone who really needed to move to a different database. If they did it usually is a political issue and not a technical one (uhh, we hired this new operations manager and MS SQL is so much better than postgresql …) Unless you develop a generic software to be sold to many clients, the choice of data storage is rarely a real concern unless someone who feels the urge to be important starts throwing words he read in a magazine at management. None of my clients ever questioned the database. They either had one which they wanted to use (i.e. they had a big iron IBM with DB2 on it plus a DBA who knew what he was doing) or they didn't care as long as it worked as expected and came with a maintenance contract of sorts.
If you're developing a web application, the least concern is portability (you're not going to replace the million lines of PHP either). The biggest concern should lie with data integrity and API's that protect your data and business logic so you can hire that cheap app development company from overseas your boss recommended (based on a spam email) without running the risk of compromising everything.
Uwe
On the other hand, when I was trying to store all my logic in a database, there was just one thing that made me hate it. Testing. Testing the procedures inside the database was not easy, not funny, and too much time consuming.--regards Szymon Lipiński
I agree that it's not like turning on the light switch. And I'm not implying there isn't a logic layer between the database and the application. Based off my past experiences I would likely not put business logic in the database unless it was a critical for performance. This does not make it portable and does the performance of my product require it? It really comes down to the application there is not one big paint brush. We have all be around and get this. I would not likely design a solution that had the database and the application layers both containing the business logic. I have seen this and the unexpected behavior as assumptions are made on both ends of that spectrum. I like to keep it simple where I can. This all being said I think database minded folks should own DAO's. I think if your a database guy and you don't own the DAO's you are missing an opportunity to really make a difference and get more aligned with your development staff. Doesn't matter what code base DAO's are in it's a repetitive pattern that any database person can pick up.On Wed, May 4, 2016 at 12:29 PM, Szymon Lipiński <mabewlun@gmail.com> wrote:I'm not sure that when a company buys another company they can just migrate the other database without any logic layer. The data is usually useless without the business layer which tells how to merge all the parts together to have a simple answer to a question like "do we have this in stock". And for such a migration that's not too important if we have the logic in database, or in some other layer. Of course it is always simpler to migrate a database treated like a CSV file, where all the logic (including constraints) is in an external application. But do we really want that?On 4 May 2016 at 18:14, Will McCormick <wmccormick@gmail.com> wrote:I agree it's typically political but so are most things business. Examples: Companies buy other companies - You are using a competitors data store and want to replace it. Company needs to compete with competitors and wants to reduce cost ... these are not technical requirements and it's certainly not vapor ideology. I have only worked for startups and have seen this happen at every company i have worked for, yes it is political but yes it happens. Smaller companies are more susceptible to it.
The reality is somewhere in the middle as it often is. My point is you don't have to replace a million lines of code if you plan upfront. If you don't .. you do.On Wed, May 4, 2016 at 11:29 AM, Uwe Schroeder <uwe@oss4u.com> wrote:On Wed, May 04, 2016 11:05:25 AM Will McCormick wrote:
A reason to consider may be portability. What happens if I want to let my customer chose their data store or I just don't want to put all my eggs in one basket.Technically there are truths but you cannot ignore the business side either. If a we can exceed our performance requirements and keep things generic/portable this is the best of both worlds.I think this is the main reason people separate the business logic from the database. How many of you have ported databases between platforms? Or had multiple types of data stores in the same company?
I have been waiting for the portability argument for the last 20+ posts :-) Everyone who did any type of consulting/working in this field knows that latest the second argument from management is “portability”. I have yet to see anyone who really needed to move to a different database. If they did it usually is a political issue and not a technical one (uhh, we hired this new operations manager and MS SQL is so much better than postgresql …) Unless you develop a generic software to be sold to many clients, the choice of data storage is rarely a real concern unless someone who feels the urge to be important starts throwing words he read in a magazine at management. None of my clients ever questioned the database. They either had one which they wanted to use (i.e. they had a big iron IBM with DB2 on it plus a DBA who knew what he was doing) or they didn't care as long as it worked as expected and came with a maintenance contract of sorts.
If you're developing a web application, the least concern is portability (you're not going to replace the million lines of PHP either). The biggest concern should lie with data integrity and API's that protect your data and business logic so you can hire that cheap app development company from overseas your boss recommended (based on a spam email) without running the risk of compromising everything.
Uwe
On the other hand, when I was trying to store all my logic in a database, there was just one thing that made me hate it. Testing. Testing the procedures inside the database was not easy, not funny, and too much time consuming.--regards Szymon Lipiński
I also wouldn't keep the logic in a database. And the DAO layer is the best solution I know. The biggest problem I experienced was that there was no DBA in a team, management didn't see any problem with that, and the DAO layer was managed by random programmers who didn't want to learn databases. The results were mainly two: slow development (due to too many bugs), and slow application (due to bad queries).On 4 May 2016 at 19:09, Will McCormick <wmccormick@gmail.com> wrote:I agree that it's not like turning on the light switch. And I'm not implying there isn't a logic layer between the database and the application. Based off my past experiences I would likely not put business logic in the database unless it was a critical for performance. This does not make it portable and does the performance of my product require it? It really comes down to the application there is not one big paint brush. We have all be around and get this. I would not likely design a solution that had the database and the application layers both containing the business logic. I have seen this and the unexpected behavior as assumptions are made on both ends of that spectrum. I like to keep it simple where I can. This all being said I think database minded folks should own DAO's. I think if your a database guy and you don't own the DAO's you are missing an opportunity to really make a difference and get more aligned with your development staff. Doesn't matter what code base DAO's are in it's a repetitive pattern that any database person can pick up.On Wed, May 4, 2016 at 12:29 PM, Szymon Lipiński <mabewlun@gmail.com> wrote:I'm not sure that when a company buys another company they can just migrate the other database without any logic layer. The data is usually useless without the business layer which tells how to merge all the parts together to have a simple answer to a question like "do we have this in stock". And for such a migration that's not too important if we have the logic in database, or in some other layer. Of course it is always simpler to migrate a database treated like a CSV file, where all the logic (including constraints) is in an external application. But do we really want that?On 4 May 2016 at 18:14, Will McCormick <wmccormick@gmail.com> wrote:I agree it's typically political but so are most things business. Examples: Companies buy other companies - You are using a competitors data store and want to replace it. Company needs to compete with competitors and wants to reduce cost ... these are not technical requirements and it's certainly not vapor ideology. I have only worked for startups and have seen this happen at every company i have worked for, yes it is political but yes it happens. Smaller companies are more susceptible to it.
The reality is somewhere in the middle as it often is. My point is you don't have to replace a million lines of code if you plan upfront. If you don't .. you do.On Wed, May 4, 2016 at 11:29 AM, Uwe Schroeder <uwe@oss4u.com> wrote:On Wed, May 04, 2016 11:05:25 AM Will McCormick wrote:
A reason to consider may be portability. What happens if I want to let my customer chose their data store or I just don't want to put all my eggs in one basket.Technically there are truths but you cannot ignore the business side either. If a we can exceed our performance requirements and keep things generic/portable this is the best of both worlds.I think this is the main reason people separate the business logic from the database. How many of you have ported databases between platforms? Or had multiple types of data stores in the same company?
I have been waiting for the portability argument for the last 20+ posts :-) Everyone who did any type of consulting/working in this field knows that latest the second argument from management is “portability”. I have yet to see anyone who really needed to move to a different database. If they did it usually is a political issue and not a technical one (uhh, we hired this new operations manager and MS SQL is so much better than postgresql …) Unless you develop a generic software to be sold to many clients, the choice of data storage is rarely a real concern unless someone who feels the urge to be important starts throwing words he read in a magazine at management. None of my clients ever questioned the database. They either had one which they wanted to use (i.e. they had a big iron IBM with DB2 on it plus a DBA who knew what he was doing) or they didn't care as long as it worked as expected and came with a maintenance contract of sorts.
If you're developing a web application, the least concern is portability (you're not going to replace the million lines of PHP either). The biggest concern should lie with data integrity and API's that protect your data and business logic so you can hire that cheap app development company from overseas your boss recommended (based on a spam email) without running the risk of compromising everything.
Uwe
On the other hand, when I was trying to store all my logic in a database, there was just one thing that made me hate it. Testing. Testing the procedures inside the database was not easy, not funny, and too much time consuming.--regards Szymon LipińskiIt seems like there is no problem with storing logic anywhere, the problem is with having someone who knows how to write it.--regards Szymon Lipiński
I 100% agree with you. It's always been a problem but it is up to us to take ownership and provide value. Some would be surprising shocked how simple it is to manage the Data access layer once the framework is in place regardless of what it is written in. For the same reasons you wouldn't typically have Application Developers configuring your production disks for high performance... why would you ever have them access the database inefficiently? There is an assumption designers are good at SQL or at least know it ... I challenge you to flip that around and learn the Data Access Layer. Companies do not knowingly spend money on hardware to have it consumed by inefficient data access? No executive signs up to increase the TCO and reduce profit margins when they could be making more money? But this is far to often the case and the root cause is they did not have the right tool (pun not intended) for the job.On Wed, May 4, 2016 at 1:33 PM, Szymon Lipiński <mabewlun@gmail.com> wrote:I also wouldn't keep the logic in a database. And the DAO layer is the best solution I know. The biggest problem I experienced was that there was no DBA in a team, management didn't see any problem with that, and the DAO layer was managed by random programmers who didn't want to learn databases. The results were mainly two: slow development (due to too many bugs), and slow application (due to bad queries).On 4 May 2016 at 19:09, Will McCormick <wmccormick@gmail.com> wrote:I agree that it's not like turning on the light switch. And I'm not implying there isn't a logic layer between the database and the application. Based off my past experiences I would likely not put business logic in the database unless it was a critical for performance. This does not make it portable and does the performance of my product require it? It really comes down to the application there is not one big paint brush. We have all be around and get this. I would not likely design a solution that had the database and the application layers both containing the business logic. I have seen this and the unexpected behavior as assumptions are made on both ends of that spectrum. I like to keep it simple where I can. This all being said I think database minded folks should own DAO's. I think if your a database guy and you don't own the DAO's you are missing an opportunity to really make a difference and get more aligned with your development staff. Doesn't matter what code base DAO's are in it's a repetitive pattern that any database person can pick up.On Wed, May 4, 2016 at 12:29 PM, Szymon Lipiński <mabewlun@gmail.com> wrote:I'm not sure that when a company buys another company they can just migrate the other database without any logic layer. The data is usually useless without the business layer which tells how to merge all the parts together to have a simple answer to a question like "do we have this in stock". And for such a migration that's not too important if we have the logic in database, or in some other layer. Of course it is always simpler to migrate a database treated like a CSV file, where all the logic (including constraints) is in an external application. But do we really want that?On 4 May 2016 at 18:14, Will McCormick <wmccormick@gmail.com> wrote:I agree it's typically political but so are most things business. Examples: Companies buy other companies - You are using a competitors data store and want to replace it. Company needs to compete with competitors and wants to reduce cost ... these are not technical requirements and it's certainly not vapor ideology. I have only worked for startups and have seen this happen at every company i have worked for, yes it is political but yes it happens. Smaller companies are more susceptible to it.
The reality is somewhere in the middle as it often is. My point is you don't have to replace a million lines of code if you plan upfront. If you don't .. you do.On Wed, May 4, 2016 at 11:29 AM, Uwe Schroeder <uwe@oss4u.com> wrote:On Wed, May 04, 2016 11:05:25 AM Will McCormick wrote:
A reason to consider may be portability. What happens if I want to let my customer chose their data store or I just don't want to put all my eggs in one basket.Technically there are truths but you cannot ignore the business side either. If a we can exceed our performance requirements and keep things generic/portable this is the best of both worlds.I think this is the main reason people separate the business logic from the database. How many of you have ported databases between platforms? Or had multiple types of data stores in the same company?
I have been waiting for the portability argument for the last 20+ posts :-) Everyone who did any type of consulting/working in this field knows that latest the second argument from management is “portability”. I have yet to see anyone who really needed to move to a different database. If they did it usually is a political issue and not a technical one (uhh, we hired this new operations manager and MS SQL is so much better than postgresql …) Unless you develop a generic software to be sold to many clients, the choice of data storage is rarely a real concern unless someone who feels the urge to be important starts throwing words he read in a magazine at management. None of my clients ever questioned the database. They either had one which they wanted to use (i.e. they had a big iron IBM with DB2 on it plus a DBA who knew what he was doing) or they didn't care as long as it worked as expected and came with a maintenance contract of sorts.
If you're developing a web application, the least concern is portability (you're not going to replace the million lines of PHP either). The biggest concern should lie with data integrity and API's that protect your data and business logic so you can hire that cheap app development company from overseas your boss recommended (based on a spam email) without running the risk of compromising everything.
Uwe
On the other hand, when I was trying to store all my logic in a database, there was just one thing that made me hate it. Testing. Testing the procedures inside the database was not easy, not funny, and too much time consuming.--regards Szymon LipińskiIt seems like there is no problem with storing logic anywhere, the problem is with having someone who knows how to write it.--regards Szymon Lipiński
You forgot that it is intended, there is the mantra "developers are expensive, hardware is cheap".
You forgot that it is intended, there is the mantra "developers are expensive, hardware is cheap".On 4 May 2016 at 20:20, Will McCormick <wmccormick@gmail.com> wrote:I 100% agree with you. It's always been a problem but it is up to us to take ownership and provide value. Some would be surprising shocked how simple it is to manage the Data access layer once the framework is in place regardless of what it is written in. For the same reasons you wouldn't typically have Application Developers configuring your production disks for high performance... why would you ever have them access the database inefficiently? There is an assumption designers are good at SQL or at least know it ... I challenge you to flip that around and learn the Data Access Layer. Companies do not knowingly spend money on hardware to have it consumed by inefficient data access? No executive signs up to increase the TCO and reduce profit margins when they could be making more money? But this is far to often the case and the root cause is they did not have the right tool (pun not intended) for the job.On Wed, May 4, 2016 at 1:33 PM, Szymon Lipiński <mabewlun@gmail.com> wrote:I also wouldn't keep the logic in a database. And the DAO layer is the best solution I know. The biggest problem I experienced was that there was no DBA in a team, management didn't see any problem with that, and the DAO layer was managed by random programmers who didn't want to learn databases. The results were mainly two: slow development (due to too many bugs), and slow application (due to bad queries).On 4 May 2016 at 19:09, Will McCormick <wmccormick@gmail.com> wrote:I agree that it's not like turning on the light switch. And I'm not implying there isn't a logic layer between the database and the application. Based off my past experiences I would likely not put business logic in the database unless it was a critical for performance. This does not make it portable and does the performance of my product require it? It really comes down to the application there is not one big paint brush. We have all be around and get this. I would not likely design a solution that had the database and the application layers both containing the business logic. I have seen this and the unexpected behavior as assumptions are made on both ends of that spectrum. I like to keep it simple where I can. This all being said I think database minded folks should own DAO's. I think if your a database guy and you don't own the DAO's you are missing an opportunity to really make a difference and get more aligned with your development staff. Doesn't matter what code base DAO's are in it's a repetitive pattern that any database person can pick up.On Wed, May 4, 2016 at 12:29 PM, Szymon Lipiński <mabewlun@gmail.com> wrote:I'm not sure that when a company buys another company they can just migrate the other database without any logic layer. The data is usually useless without the business layer which tells how to merge all the parts together to have a simple answer to a question like "do we have this in stock". And for such a migration that's not too important if we have the logic in database, or in some other layer. Of course it is always simpler to migrate a database treated like a CSV file, where all the logic (including constraints) is in an external application. But do we really want that?On 4 May 2016 at 18:14, Will McCormick <wmccormick@gmail.com> wrote:I agree it's typically political but so are most things business. Examples: Companies buy other companies - You are using a competitors data store and want to replace it. Company needs to compete with competitors and wants to reduce cost ... these are not technical requirements and it's certainly not vapor ideology. I have only worked for startups and have seen this happen at every company i have worked for, yes it is political but yes it happens. Smaller companies are more susceptible to it.
The reality is somewhere in the middle as it often is. My point is you don't have to replace a million lines of code if you plan upfront. If you don't .. you do.On Wed, May 4, 2016 at 11:29 AM, Uwe Schroeder <uwe@oss4u.com> wrote:On Wed, May 04, 2016 11:05:25 AM Will McCormick wrote:
A reason to consider may be portability. What happens if I want to let my customer chose their data store or I just don't want to put all my eggs in one basket.Technically there are truths but you cannot ignore the business side either. If a we can exceed our performance requirements and keep things generic/portable this is the best of both worlds.I think this is the main reason people separate the business logic from the database. How many of you have ported databases between platforms? Or had multiple types of data stores in the same company?
I have been waiting for the portability argument for the last 20+ posts :-) Everyone who did any type of consulting/working in this field knows that latest the second argument from management is “portability”. I have yet to see anyone who really needed to move to a different database. If they did it usually is a political issue and not a technical one (uhh, we hired this new operations manager and MS SQL is so much better than postgresql …) Unless you develop a generic software to be sold to many clients, the choice of data storage is rarely a real concern unless someone who feels the urge to be important starts throwing words he read in a magazine at management. None of my clients ever questioned the database. They either had one which they wanted to use (i.e. they had a big iron IBM with DB2 on it plus a DBA who knew what he was doing) or they didn't care as long as it worked as expected and came with a maintenance contract of sorts.
If you're developing a web application, the least concern is portability (you're not going to replace the million lines of PHP either). The biggest concern should lie with data integrity and API's that protect your data and business logic so you can hire that cheap app development company from overseas your boss recommended (based on a spam email) without running the risk of compromising everything.
Uwe
On the other hand, when I was trying to store all my logic in a database, there was just one thing that made me hate it. Testing. Testing the procedures inside the database was not easy, not funny, and too much time consuming.--regards Szymon LipińskiIt seems like there is no problem with storing logic anywhere, the problem is with having someone who knows how to write it.--regards Szymon Lipiński--regards Szymon Lipiński