Data Privacy Regulations and Automation of Data Privacy

 Automation of Data Privacy: 

You have all this data available in your data stores or is available in third party services. Before you build out your business services, you should be familiar with what type of data you are allowed to store, share or use for your clients. Depending on your business' locale and or your clients locale, you will be required to understand the laws and regulations around data acquisition, use, and dissemination. Most of the rules that regulate data revolve around data privacy to protect individual, organizations and businesses.

For example, firms doing business in California have to follow CPRA (California Privacy Regulations Act) and US Privacy rules.  You have equivalent counterparts in other geographical regions such as GDPR (General Data Protection Rules ) for the Europe Union. 

Why should a business care about privacy and privacy rules? It is crucial for users to trust the parties they engage in business with to safeguard their information from bad players who engage in fraud and theft.  This saves users or businesses from oversharing sensitive information that could lead to loss of assets or reputation. 

Applying the rules:

It takes specialists in the regulatory law to understand all the regulations. And, business specialists to translate those regulations into business terms. Therefore, the rules being applied should apply to the data you have or want to acquire and the business you are engaged in. 

Define your business requirements.

The process is iterative as rules and business requirements change. To kick start the process, you will need to start with very general business requirements such as type of business you are engaged in and location of your business and clients. You should understand how data will influence your business. Therefore, some technical savviness can bring success.

Data asset identification, categorization and classification.

Both business and technical professionals need to create a data catalog to identify data categories and business classification. What is the difference between data classification and categorization. Categorization refers to technical data attributes such as as id, url, IP address, name etc. Classification refers to the business use of the data such as audio, visual, PII, metadata etc.

For example media data categories can be graphical, audio-visual etc. And these can be classified further into human identifiable, protected or unprotected data. Some privacy rules and intellectual property rules may apply to human identifiable data and protected data respectively.

Transactional data often be classified into financial data, PII or other metadata. These may require different rules to use, store and share. For example, an organization may require auditors to have access to financial transactions, but Privacy laws may require PII to not be shared along with financial data to 3rd parties. Other jurisdictions may find certain attributes in metadata to be PII such as IP address or cookie information. Therefore metadata may need to be reclassification for different jurisdictions. 

Documentation can be a data dictionary which maps data attributes to human readable description; their business classification; and location such as database or 3rd party API service.

Rule mapping.

Rules governing your data assets can be mapped to the catalogs’ classifications and subsequent categories. For example mapping potential jurisdiction overlaps and their PII or metadata (IP addresses or cookies).  

Another example is mapping jurisdiction rules governing facial recognition in photographs or videos captured public areas.

Regulatory and Business professionals need to collaborate here to complete classification to rules mappings. Technical professionals can refine the rules mapping to the categories if needed.

You can use the data dictionary to construct a score attributed to a rule.

Impact assessment.

What impacts do the rules have on your business?

Also, what rules require special attention or resources in order to remain certifiable or trustworthy? 

Impact assessment can be captured graphically as score or color code on data classifications. A summary or business-wide Impact Assessment can also be derived for each data privacy rule. 

You can continue to refine the impact assessment by revising the rules, classifications and categories. 

Prescription analysis:

Impact assessment can help drive the execution of the rules by narrowing down problem areas and prescribed solutions. Prescribed solutions are a continuous process as upstream processes and requirements continue to change.

How much change do you intend to make in each cycle? This happens to be dependent on how much is invested in each cycle and how many cycles or iterations you plan to have. Thus driven by business planning and by engineering capabilities. 

Automating the rules:

Manual execution the rules can be done during analysis.

Automation may require some more planning and design from technology specialists. Some considerations for the technology stack are required. For example, PII may need to be abstracted at the data layer during persistence or upon request. Here, action identification or data stream processing can trigger a prescriptive action such as logging a warning, throwing an exception, or url redirection. 

Authorization layers should be put in place to limit data visibility and augmentation. 

Data filters can be inserted into API requets to limit certain columns and also limit data request volumes. 

AI and ML filters can be implemented to supervise user  behavior and detect unscrupulous requests..

On Data Governance 

The purpose of data governance is to provide a data driven business with assurance of regulatory certification. Certification is both internal and external. Internal certification is defined by the business itself to safeguard proprietary information. External certification are by overseers such as jurisdiction, consortiums and investors. US privacy laws are jurisdiction requirements while ISO standards are expert/industry driven in order to provide a blueprint for data interchangeable. Investor standards tend to fluctuate depending on investors’ culture, such as environmental issues or political standing on current issues.

 

Analytics with Machine learning

How can an organization get into the data resale business game?

In order to have the capability to perform analytics using automated processes, you will be required to have the ability to perform: 

1. Descriptive analytics

2. Diagnostic analytics

3. Predictive analytics

4. Prescriptive analytics

using qualitative and quantitative methods.

Descriptive analytics - Begins with determining key performance measures of your business. Then you can employ BI tools and data-mining capabilities on these performance measures to determine past and current events. 

Diagnostic analytics - This is also called forensics analytics where you determine why good or bad things happened. This can be achieved through pattern extraction from the events identified in descriptive analytics phase.

Predictive analytics -  Here, business models are created to simulate pattern of events discovered in the diagnostics phase. These models are used to extrapolate future events. 

Prescriptive analytics - This phase optimizes the performance key measures by putting into consideration the prediction models and business goals; for example minimizing response latency or maximized public reach.

Data is not always precisely organized especially when dealing with data collected from many sources. Therefore, before beginning data transformation you have to categorize your raw data as complete data; incomplete data; or fragmented data.

• Complete data - can be personal identifiable and contacts information.

• Incomplete data - can be purchase history such as consumer data.

• Fragmented data - this can include crawled data and social data.

You can then transform the data by adding value to raw data by filling in data gaps and highlighting embedded information. This is accomplished by processing fragmented and incomplete data into a common data repository. The final product serves as foundation data for your clients to gain knowledge and make better decisions.

For the clients, the cost of acquiring a complete data set is minimized because you will have absolved the overall infrastructural cost i.e. the cost of collecting; processing; and storing data. 

Automated Data Processing.

The idea is to process data from different sources from input to storage automatically. Data is fed into the system in the different forms the source allows: i.e. bulk, streams, messaging etc.

Given a source of raw data an analyst can recommend collection of specific data elements which can be stored as a configuration. Multiple configurations are required so as to correspond with respective data sources and policies. 

Standardized data is then contextualized by matching against indexes or 3rd party services. For example a tweethandle-to-name can be matched against consumer data or an email feed from a third party.

Adding data to the repository requires verification of duplicate data and some business use cases to decide on whether to insert or update data. 

Machine learning and data processing

Where does machine learning fall into all this?

Data source description and corresponding data reformulation will require keeping memory of past events, thus the learning process. Learning can be manual, semiautomated, or fully automated. An example of manual learning is an analyst doing a study and updating the formulation/configuration. A semi-autonomous learning process can involve supervised learning of source classifications. Fully automated learning can be an evolution of the supervised learning or complete arbitrary classification of data sources through online learning. 

Data matching can also benefit from machine learning. To eliminate the network cost between matching process and indexing engines, a smart matcher can be implemented. This matcher can be created from a learning process to classify some data elements and can be loaded in memory during execution time.