0 - 40 of 40 tags for v_extensibility

 

Overview

This article provides an example of implementing a java table operator. The specific use case is support for the SHA-2 family of hash encodings, for more details on SHA-2 see https://en.wikipedia.org/wiki/SHA-2

Background

This article is a follow on to article [1] which discussed implementing K-means using a Teradata release 14.10 table operator. The main contribution of this article is to discuss how to use the new Teradata 15.0 multiple input stream feature and a short discussion on a gcc compiler performance optimization.

XML is a markup language, used to format data in a wide variety of applications. It is commonly used as a message format for application integration (e.g. XML messages exchanged between applications, with those applications implementing an XML based API).

This article describes how to use Teradata query grid to execute a Mahout machine learning algorithm on a Hadoop cluster based on data sourced from the Teradata Integrated Data Warehouse.

Querying XML


Stored Procedures


SQL Stored Procedures were added to Teradata around 2003 with the release of Teradata V2R5.1. Since then the capabilities of SQL Stored Procedure s has been expanded. However, there are still some queries that cannot be run directly from within a Stored Procedure.

 

In a prior article [1] we described how to use the Teradata 14.10 CalcMatrix operator and R to perform a multiple variable linear regression analysis. This article extends that concept with a comprehensive in database solution by introducing a new in database table operator named “CM_Solve”.

Linear Regression

In statistics, linear regression is an approach to model the relationship between a scalar dependent variable y and one or more independent variables denoted x. Linear regression is one of the oldest and most fundamental types of analysis in statistics. The British scientist Sir Francis Galton originally developed it in the latter part of the 19th century. The term "regression" derives from the nature of his original study in which he found that the children of both tall and short parents tend to "revert" or "regress" toward average heights.

Table Operators

This article discusses how to implement a Teradata 14.10 table operator using K-means clustering as an example use case. 

As part of Teradata Database 14.10 Teradata Intelligent Memory (TIM) feature, a table function was developed to provide the heatmap report for a specific AMP for database object(s) for systems in which data temperature collection is enabled[1].     This HeatMap table function is being provided fo

SQL provides a set of useful functions, but they might not satisfy all of the particular requirements you have to process your data.

User-defined functions (UDFs) allow you to extend SQL by writing your own functions in the C/C++ programming languages, installing them on the database, and then using them like standard SQL functions.

Customers are currently leveraging Teradata's features such as User Defined Functions (UDFs) and External Stored Procedures (XSPs) to develop complex applications.

The aim of this set articles is to show the user how to use the Eclipse IDE to debug in a single-node, controlled, non-production environment:

  • Java or C/C++ User Defined Functions (UDFs)
  • Java or C/C++ External Stored Procedures (XSPs)

The audience for this set of articles ranges from Teradata associates within the R&D and Professional Services organizations to  Teradata Customer and Partner developers who want to debug a UDF/XSP running on their  DBS VM before installing it on a production system.

Virtual Network Computing (VNC) is a graphical desktop sharing system that uses the RFB protocol (remote framebuffer) to remotely control another computer. It transmits the keyboard and mouse events from one computer to another, relaying the graphical screen updates back in the other direction, over a network.

In addition to stored procedures, which use SQL control and condition-handling statements, input and output parameters, and local variables to provide applications with a server-based, precompiled procedural interface, Teradata Database supports external stored procedures.

SQL provides a set of useful functions, but they might not satisfy all of the particular requirements you have to process your data.

User-defined functions (UDFs) allow you to extend SQL by writing your own functions in the C/C++ programming languages, installing them on the database, and then using them like standard SQL functions.

SQL provides a set of useful functions, but they might not satisfy all of the particular requirements you have to process your data.

User-defined functions (UDFs) allow you to extend SQL by writing your own functions in the C/C++ programming languages, installing them on the database, and then using them like standard SQL functions.

SQL provides a set of useful functions, but they might not satisfy all of the particular requirements you have to process your data.

User-defined functions (UDFs) allow you to extend SQL by writing your own functions in the C/C++ programming languages, installing them on the database, and then using them like standard SQL functions.

SQL provides a set of useful functions, but they might not satisfy all of the particular requirements you have to process your data.

User-defined functions (UDFs) allow you to extend SQL by writing your own functions in the C/C++ programming languages, installing them on the database, and then using them like standard SQL functions.

In addition to stored procedures, which use SQL control and condition-handling statements, input and output parameters, and local variables to provide applications with a server-based, precompiled procedural interface, Teradata Database supports external stored procedures.

SQL provides a set of useful functions, but they might not satisfy all of the particular requirements you have to process your data.

User-defined functions (UDFs) allow you to extend SQL by writing your own functions in the Java programming languages, installing them on the database, and then using them like standard SQL functions.

SQL provides a set of useful functions, but they might not satisfy all of the particular requirements you have to process your data.

User-defined functions (UDFs) allow you to extend SQL by writing your own functions in the Java programming language, installing them on the database, and then using them like standard SQL functions.

SQL provides a set of useful functions, but they might not satisfy all of the particular requirements you have to process your data.

User-defined functions (UDFs) allow you to extend SQL by writing your own functions in the Java programming language, installing them on the database, and then using them like standard SQL functions.

An integrated development environment is crucial to software developers for editing, building and debugging their program all within one environment. Therefore, setting up the right development environment is an important first task.

Since Teradata first introduced native Geospatial capabilities in the database there have been inquiries about how to interoperate with the spatial data via client GIS and visualization tools.  In this article we are going to discuss how to view and edit spatial features and attributes maintained within Teradata in real-time by using the GeoServer web services.

As most of you might agree, managing our collections of digitial pictures is becoming quite a challenge.  The number of photos continues to increase and now includes pictures from cameras as well as multiple mobile devices.  And to add to my troubles, I find that I have duplicate copies in different folders and on different computers.  Getting this organized is becoming a high priority.  Sure there are management solutions already available, but hey, we're tech people and it's more fun to try to build our own!  With the free Teradata Express database and some java coding, we have the right tools to get started.

One of the new compression features in Teradata 13.10 is Block Level Compression (BLC), which provides the capability to perform compression on whole data blocks at the file system level before the data blocks are actually written to storage. Like any compression features, BLC helps save space and reduce I/O. 

There is a CPU cost to perform compression on inserting data. And there is a CPU cost to perform decompression on whole data blocks whenever the compressed data blocks are accessed. Even when only one column of a single row is needed, the whole data block must be decompressed. For updates, the compressed data blocks have to be decompressed first and then recompressed. Careful evaluations shall be done before applying BLC in your production systems.

Hadoop MapReduce programmers often find that it is more convenient and productive to have direct access from their MapReduce programs to data stored in a RDBMS such as Teradata Enterprise Data Warehouse (EDW) because:

  1. There is no benefit to exporting relational data into a flat file.
  2. There is no need to upload the file into the Hadoop Distributed File System (HDFS).
  3. There is no need to change and rerun the scripts/commands in the first two steps when they need to use different tables/columns in their MapReduce programs.

As Teradata customers discover and begin to utilize the native Teradata database geospatial capabilities, one of the first questions that inevitably comes up is, how do I “Geocode” my data?  In fact, Geocoding will often be an important first phase of any Geospatial implementation project and sometimes even a barrier to start the project all together.  The purpose of this article is to discuss what Geocoding is, how it works, Geocoding options, precision, and sources available today for Geocoded information.

Teradata 13.10 provides Algorithmic Compression (ALC) feature that allows  users to apply compression / decompression functions on a specific column of character or byte type. The compression / decompression functions may be Teradata built-in functions provided along with ALC or user provided compression / decompression algorithm registered as UDFs.

Teradata has added geospatial features to Teradata 13 (and earlier versions with the optional extension package - see my earlier article here).  These features enable powerful location based analytics, but often I'm asked how to get start

Someone asked a few days ago for an easier and quicker way to calculate distance between two points on a sphere without having to transform to the UTM SRS (Spatial Reference System) from the WGS84 SRS.

The availability of Teradata's geospatial extension package in 2007 brought these location capabilities to Teradata 12, 6.2 and 6.1.  This package is still available as a free download from Teradata and when installed, adds geospatial functionality as a User Defined Type (UDT) along with a library of User Defined Functions (UDFs).  (

Hadoop systems [1], sometimes called Map Reduce, can coexist with the Teradata Data Warehouse allowing each subsystem to be used for its core strength when solving business problems. Integrating the Teradata Database with Hadoop turns out to be straight forward using existing Teradata utilities and SQL capabilities. There are a few options for directly integrating data from a Hadoop Distributed File System (HDFS) with a Teradata Enterprise Data Warehouse (EDW), including using SQL and Fastload. This document focuses on using a Table Function UDF to both access and load HDFS data into the Teradata EDW. In our examples, there is historical data already in Teradata EDW, presumably derived from HDFS for trend analysis. We will show examples where the Table Function UDF approach is used to perform inserts or joins from HDFS with the data warehouse.

Map-reduce, or its open source version Hadoop, is a parallel programming framework for running scripts, Java, C, and other external programming languages on hundreds of nodes. It is popular with Dot.Com companies who have large server farms and need to produce reports on website activity or produce search indexes. In general, Map-reduce applications overlap BI applications and data warehouses. However, Map-reduce applications can coexist with a data warehouse: one parallel processing, the other parallel database. Coexistence allows each subsystem’s best capabilities to be used to complement the other. With Teradata’s in-database processing technology, Map-reduce can become MPP ETL subsystem, or we can run Map-Reduce functions inside the EDW, or using table functions we can directly integrate with the Map-reduce nodes. This article illustrates a commonly used Map-reduce function running inside the Teradata EDW.

From time to time I’m asked how one might go about writing an UDF or an INMod or some other procedural extension to the database. The question isn’t a “where do I learn C or C++” question; rather it is a how do I go about debugging and testing my extension without the overheads and constraints of running within the database (or utility).


My response is you can use “my Teradata” for which I have the source code (attached). This will allow you to run, test and debug your function outside of Teradata. In short with “my Teradata” all of the features of your IDE will be available to you, including your debugger.

In my previous article we explored Teradata's spatial features using geometric shapes on a 10x10 grid.  I hope this was a useful introduction into the new ST_Geometry data type and its powerful set of functions.  And now that we've covered the basics, let's t

Now that you’ve installed Teradata’s geospatial features with Teradata Express let’s roll up our sleeves and see what we can do with this. (If you haven’t setup your TD Express with Geospatial yet, see my article.

We’re in the midst of an explosion in the world of location data. GPS tools have greatly expanded our ability to capture location data and mapping tools have broadly expanded its use. We have Web navigation sites like Google Earth and Yahoo Maps. Our cars have navigation devices to keep us from getting lost.

Teradata XML Services (TXS) extends the database’s XML capabilities focusing on transforming data between its XML and relational representations. Two key components of XML Services are:

This user guide describes the Teradata Warehouse 7.1 User Defined Function (UDF) feature. This guide is primarily targeted towards developers of UDFs and database administrators.