Applying Common Protections Across Tables
IRI's data-centric security software, FieldShield, is being updated to v3.1. This is a major new release that allows DBAs using FieldShield in the IRI Workbench to apply common masking, encryption, pseudonymization or other data privacy functions to similar columns with personally identifying information (PII) across multiple tables within one or more database schemas. This profiling functionality will be extended to CoSort ETL users in the IRI Workbench wishing to apply common data transformation rules to many sources at once.
FieldShield's data protection rule engine allows users to specify a regular expression identifying the columns in selected tables that will receive a common security function. For example, the same format-preserving encryption algorithm and encryption key can be applied to every social security number column in every table, even if the column names were slightly different. A single job definition for encryption and decryption for an entire database is not only convenient, but preserves referential integrity by keeping the intra-table data encrypted, and thus linked, in the same way.
In addition to rule-based protections, FieldShield v3 features new protection functions, such as:
FIPS-compliant OpenSSL and 3DES encryption
AES-128 (in addition to AES-256 with or without format preservation)
Random selection or data generation
Custom data masks
Field shifting and more string manipulations
Other features, including row- and column-level filters, reversible and non-reversible pseudonymization, and near-value lookup file logic, further enhance FieldShield's ability to protect PII in static and changing dimensions. If you would like to test FieldShield v3, click here to send your request with the details needed.
IRI Expands in Eurasia
New Partner Offices in Russia and Turkey
New resale and support partners will expand the reach of IRI's growing software product line:
Aksioma Group was founded in 2010 by a team of IT professionals aiming at the realization of complex IT projects for major companies in Russia. Today the Moscow-based company focuses on consulting, infrastructure solutions, and information security, making it a perfect fit for offering licensing and implementation support for IRI's big data processing and data-centric protection tools. Aksioma's first joint customer with IRI and Sabre, Russian Railways, uses CoSort to accelerate database loads and analyze traffic data to discover the most profitable, brand-friendly mix of scheduling, price and service. Using CoSort in applications like these will continue to add business value to products and projects developed in Russia that Aksioma and its partners can realize.
Asteroit Teknoloji provides companies in Turkey with short- and long-term consulting services for data warehouses, business intelligence, data and ICT services. Based in Istanbul, Asteroit has expertise in data modeling, DW design and development, CRM, Oracle DB management and performance tuning, and GSM technologies for telco and mobile carriers. FACT and CoSort will help Asteroit customers accelerate the extraction, transformation, and loading (ETL) of CDR and other big data in Oracle and beyond. IRI's David Friedland is visiting Asteroit this month to provide training on the IRI Workbench, currently supporting CoSort 9.5.2, FACT 2.5.1, FieldShield 3.1.0 (beta), and RowGen 3.0.1 (alpha).
NextForm Feature Advice for v3
What Are Your Data Migration Challenges?
For the last several years, IRI has offered a basic file-format interchange, data type conversion, and record layout reformatting tool called NextForm. NextForm is another commercial spin-off of the CoSort Sort Control Language (SortCL) program and will be part of the IRI Workbench (Eclipse GUI). From the information IRI has gleaned from thousands of free trial downloads, most NextForm users have been interested in high-volume LDIF-to-CSV and COBOL-text file conversion, EBCDIC-to-ASCII, and packed decimal-to-numeric data type translations.
Tech Tip - Slowly Changing Dimensions
The next release of NextForm in 2013 will introduce the ability to switch between big and little endian designations at both the file and field level, convert between native CJK data types and Unicode, and migrate between databases and certain database versions. What IRI needs to know now from those with data migration challenges are the additional areas of structured data conversions that are of interest, and what specific database migrations are planned.
Anyone who has evaluated NextForm in the past, whether or not they have licensed the product, will be entitled to a free copy of the next release in exchange for feedback before the end of 2012. Think about your current and future data migration requirements, file formats and database platforms, and what you would like to specifically have in a purpose-built tool for converting big data on Unix, Linux, and Windows. Please email firstname.lastname@example.org with your specific requirements and suggestions for NextForm and get a free upgrade!
Reporting with Fuzzy Logic Features in SortCL
The purpose of reporting on Slowly Changing Dimensions (SCD) is to find a value that can satisfy an other-than-equal relationship to the search argument. CoSort's SortCL program accomplishes this with new fuzzy table look up logic. This is an expansion of the search of a set file wherein the input field provides the key for the search.
Consider, for example, a transaction file with non-uniform intervals of increasing dates, each associated with a product price change. With or without a specific date entry, you can find the price of the product before, on, or after that date. Given the transaction file "days.in",
and the tab-delimited dimensions file "butter.set" providing the slowly changing dimensions of the price of butter on specific days,
this SortCL script finds the active price for each date in "days.in":
/REPORT # do not sort
/FIELD=(Price, SET GE butter.set[Day] \
The target file, buttercost.out, shows the effective cost on each transaction date:
default = "XX", POSITION=14, SIZE=4)
Notice that when comparing the transaction (fact) table with the limited dimensional (price) information,
the price of butter on May 19 is from May 14
there was no value to lookup on May 11
the price of butter on May 28 is a match
the price of butter on June 10 is from June 28
By default, where there is no information, no supposition is made. However, SortCL will use the last piece of available information.