GDPR, cloud and the DBA

It seams that the world is waking to the power of data.  Since the birth of the database just about every organisation has been storing data away on all aspects of its business.  This data has traditionally been used in a relational method, i.e we pull that data only when we need to look further at it.

However it seams we are now heading for a digital revolution.  For the first time we are starting to see this data being correctly mined, and used to its full value.  Its no wonder then that governments are trying to control the data we hold.

We are sat right on the edge of a time when computers can make complex decisions on how we might behave, how we spend, our honesty and our habits. Its an exciting time, but we are still yet to find a balance between allowing computers to help make our lives easier and allowing them to decisions they should not.

What we as DBAs need to ensure is that data is anonymized for compliance, but still has the ability to be mined.  This means losing personal and identifiable data, but not losing the dates meaning.  So for example, where we scramble a date of birth, we change it to within the same age bracket.  Should we change an address we keep the anonymized address in the same region. 

The shape shifting DBA

Introduction

About every 2 years a technology comes along that the rest of IT believes will wipe out the DBA.  We have seen MySQL, NoSQL, Big data and block chain.  All these have their place, but none replace the RDBMS.  And all need some kind of maintenance. 

The fact is there is no one toll that does it all, there will always need to be a mix of technologies working together to get the right data for the organisation.  But the DBA must move with the times.

Almost from the outset of the use of databases, the relation modal was adopted, and has largely remained that way for 50 years.  This put the data in the hands of the DBA, the developer and BI to serve the data.

That’s all changing, data now is being understood as the commodity it is, the heart of most companies.

What does a DBA do?

The day to day role of the on premises DBA for a RDBMS traditionally fits into these categories. 

Category	DBA responsibility
Application	Designing schema, access patterns, locking strategy, SQL development, and tuning.
	Deploying change scripts.
	Optimizing application and end-user queries (reactive tuning)
	Archiving data
	Generating needed ad hoc reports by querying from the database
	Proactive performance tuning
Access	Enrolling users and maintaining system security
	Controlling user access to the database
	Locking down host access
	Securing database privileged credentials
Database	Parameter configuration and tuning
	Cache management
	Job scheduling
Monitoring	Monitoring performance metrics, response times, and request rates
	Alerting
	Object access
	Logs
Platform	Ensuring compliance with database vendor license agreement
	Allocating system storage and planning future storage requirements for the database system
	Installing and upgrading the database software
	Performing data backups
	Patching the software that powers your database
	Troubleshooting DB errors and potentially contacting vendors for technical support
HA / DR	Ensuring that database are highly available in a system fail.
	Ensuring that databases are available in a disaster that affects entire systems.
	Monitoring performance of performance of DR / HA
	Ensuring that users and jobs are available at DR

So what’s changing? 

The role of the DBA will no longer just sit within the database management system.  The DBA will be required to understand the flow of data though the organisation.  The DBA will need to be involved with the database design from POC to production.

In the past the DBA has only concerned himself with the data that sits within the boundaries of the RDBMS, However as the need for data grows, data no longer remains within these boundaries. 

Data can now be seen being moved to big data stores, Data manipulation tools like PowerBI, data can be replicated to 100s of servers around the world, quires should be able to retrieve this data in seconds.

Why is it changing?

So in a world that is quickly requiring data at rapid rates it’s Ironic that the traditional custodian of the data is feeling left behind, in many cases they are the only people that have full access to the data and are often the only people who know where all the data sits. 

The power good data can bring is now being fully understood as BI, data analysis’s, Data scientists, developers and even the business are getting a better understating of the data held and requiring ever growing access to it.  This is also reflected in the agile development style that’s becoming so popular.  The DBA can no longer fully control data.

So are DBAs redundant in the new world

Far from it, a recent career study showed that 70% of DBAs are 45 years or older and 20% are within 10 years of retirement.  We could well be facing a DBA shortage in coming years. 

What are the roles of a DAB in the future?

The DBA now should not be thought of as the owner of the container that the data sits in.  But the owner of the flow of data though the systems, regardless of where they sit.

A lot of the same roles exist in the new world, but there is also a great deal of other tasks that should naturally sit with DBAs.

The below shows the tasks in a world with these new technologies including cloud.

Category	DBA responsibility
Cost and Vendor Management	Keeping data relevant in a world where data size = cost
	Vendor management
	Understanding of tools used by vendor and how they benefit
	Data flow from vendor to premises
Performance	Ensuring that data access sis fast and reliable
	Ensuring data being pulled is from the cloud is not affecting network performance
	Building data pipelines
Security	Have the right people got access
	Monitoring users access
	Ensuring access is revoked / evoked
	Monitor internal and external threats
Application	Designing schema, access patterns, locking strategy, SQL development, and tuning.
	Deploying change scripts.
	Optimizing application and end-user queries (reactive tuning)
	Archiving data
	Generating needed ad hoc reports by querying from the database
	Proactive performance tuning
Database Monitoring	Parameter configuration and tuning
	Cache management
	Job scheduling
	Monitoring performance metrics, response times, and request rates
Platform	Ensuring compliance with database vendor license agreement
	Allocating system storage and planning future storage requirements for the database system
	Installing and upgrading the database software
	Performing data backups
	Patching the software that powers your database
	Troubleshooting DB errors and potentially contacting vendors for technical support
Fault Tolerance and DR
	Ensuring that database are highly available in a system fail.
	Ensuring that the vendors DR/HA work for the organization
	Monitoring performance of performance of DR / HA
	DR testing
	Understanding what applications require access to cloud data

Conclusion

The DBAs still very much have a place, but have to learn to adapt to the changing world of data, how data is used and by who.