Run Vantage Express on AWS

Author: Adam Tworkiewicz
Last updated: December 12th, 2022

Overview

This how-to demonstrates how to run Vantage Express on AWS. Vantage Express is a small footprint configuration that contains a fully functional Teradata SQL Engine.

Cloud charges

Vantage Express is distributed as a virtual machine image. This how-to uses the EC2 c5n.metal instance type. It’s a bare metal instance that costs over $3/h.

If you want a cheaper option, try Google Cloud and Azure which support nested virtualization and can run Vantage Express on cheap VM’s.

If you do not wish to pay for cloud usage at all, install Vantage Express locally using VMware, VirtualBox, or UTM.

Prerequisites

An AWS account. If you need to create a new account follow the official AWS instructions.
awscli command line utility installed and configured on your machine. You can find installation instructions here: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html.

Installation

You will need a VPC with an Internet-facing subnet. If you don’t have one available, here is how you can create it:

# Copied from https://cloudaffaire.com/how-to-create-a-custom-vpc-using-aws-cli/

# Create VPC
AWS_VPC_ID=$(aws ec2 create-vpc \
  --cidr-block 10.0.0.0/16 \
  --query 'Vpc.{VpcId:VpcId}' \
  --output text)

# Enable DNS hostname for your VPC
aws ec2 modify-vpc-attribute \
  --vpc-id $AWS_VPC_ID \
  --enable-dns-hostnames "{\"Value\":true}"

# Create a public subnet
AWS_SUBNET_PUBLIC_ID=$(aws ec2 create-subnet \
  --vpc-id $AWS_VPC_ID --cidr-block 10.0.1.0/24 \
  --query 'Subnet.{SubnetId:SubnetId}' \
  --output text)

# Enable Auto-assign Public IP on Public Subnet
aws ec2 modify-subnet-attribute \
  --subnet-id $AWS_SUBNET_PUBLIC_ID \
  --map-public-ip-on-launch

# Create an Internet Gateway
AWS_INTERNET_GATEWAY_ID=$(aws ec2 create-internet-gateway \
  --query 'InternetGateway.{InternetGatewayId:InternetGatewayId}' \
  --output text)

# Attach Internet gateway to your VPC
aws ec2 attach-internet-gateway \
  --vpc-id $AWS_VPC_ID \
  --internet-gateway-id $AWS_INTERNET_GATEWAY_ID

# Create a route table
AWS_CUSTOM_ROUTE_TABLE_ID=$(aws ec2 create-route-table \
  --vpc-id $AWS_VPC_ID \
  --query 'RouteTable.{RouteTableId:RouteTableId}' \
  --output text )

# Create route to Internet Gateway
aws ec2 create-route \
  --route-table-id $AWS_CUSTOM_ROUTE_TABLE_ID \
  --destination-cidr-block 0.0.0.0/0 \
  --gateway-id $AWS_INTERNET_GATEWAY_ID \
  --output text

# Associate the public subnet with route table
AWS_ROUTE_TABLE_ASSOID=$(aws ec2 associate-route-table  \
  --subnet-id $AWS_SUBNET_PUBLIC_ID \
  --route-table-id $AWS_CUSTOM_ROUTE_TABLE_ID \
  --output text | head -1)

# Create a security group
aws ec2 create-security-group \
  --vpc-id $AWS_VPC_ID \
  --group-name myvpc-security-group \
  --description 'My VPC non default security group' \
  --output text

# Get security group ID's
AWS_DEFAULT_SECURITY_GROUP_ID=$(aws ec2 describe-security-groups \
  --filters "Name=vpc-id,Values=$AWS_VPC_ID" \
  --query 'SecurityGroups[?GroupName == `default`].GroupId' \
  --output text) &&
  AWS_CUSTOM_SECURITY_GROUP_ID=$(aws ec2 describe-security-groups \
  --filters "Name=vpc-id,Values=$AWS_VPC_ID" \
  --query 'SecurityGroups[?GroupName == `myvpc-security-group`].GroupId' \
  --output text)

# Create security group ingress rules
aws ec2 authorize-security-group-ingress \
  --group-id $AWS_CUSTOM_SECURITY_GROUP_ID \
  --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "Allow SSH"}]}]' \
  --output text

# Add a tag to the VPC
aws ec2 create-tags \
  --resources $AWS_VPC_ID \
  --tags "Key=Name,Value=vantage-express-vpc"

# Add a tag to public subnet
aws ec2 create-tags \
  --resources $AWS_SUBNET_PUBLIC_ID \
  --tags "Key=Name,Value=vantage-express-vpc-public-subnet"

# Add a tag to the Internet-Gateway
aws ec2 create-tags \
  --resources $AWS_INTERNET_GATEWAY_ID \
  --tags "Key=Name,Value=vantage-express-vpc-internet-gateway"

# Add a tag to the default route table
AWS_DEFAULT_ROUTE_TABLE_ID=$(aws ec2 describe-route-tables \
  --filters "Name=vpc-id,Values=$AWS_VPC_ID" \
  --query 'RouteTables[?Associations[0].Main != `false`].RouteTableId' \
  --output text) &&
  aws ec2 create-tags \
  --resources $AWS_DEFAULT_ROUTE_TABLE_ID \
  --tags "Key=Name,Value=vantage-express-vpc-default-route-table"

# Add a tag to the public route table
aws ec2 create-tags \
  --resources $AWS_CUSTOM_ROUTE_TABLE_ID \
  --tags "Key=Name,Value=vantage-express-vpc-public-route-table"

# Add a tags to security groups
aws ec2 create-tags \
  --resources $AWS_CUSTOM_SECURITY_GROUP_ID \
  --tags "Key=Name,Value=vantage-express-vpc-security-group" &&
  aws ec2 create-tags \
  --resources $AWS_DEFAULT_SECURITY_GROUP_ID \
  --tags "Key=Name,Value=vantage-express-vpc-default-security-group"

To create a VM you will need an ssh key pair. If you don’t have it already, create one:

aws ec2 create-key-pair --key-name vantage-key --query 'KeyMaterial' --output text > vantage-key.pem

Restrict access to the private key. Replace <path_to_private_key_file> with the private key path returned by the previous command:
```
chmod 600 vantage-key.pem
```

Get the AMI id of the latest Ubuntu image in your region:

AWS_AMI_ID=$(aws ec2 describe-images \
  --filters 'Name=name,Values=ubuntu/images/hvm-ssd/ubuntu-*amd64*' \
  --query 'Images[*].[Name,ImageId,CreationDate]' --output text \
  | sort -k3 -r | head -n1 | cut -f 2)

Create a Ubuntu VM with 4 CPU’s and 8GB of RAM, and a 70GB disk.

AWS_INSTANCE_ID=$(aws ec2 run-instances \
  --image-id $AWS_AMI_ID \
  --count 1 \
  --instance-type c5n.metal \
  --block-device-mapping DeviceName=/dev/sda1,Ebs={VolumeSize=70} \
  --key-name vantage-key \
  --security-group-ids $AWS_CUSTOM_SECURITY_GROUP_ID \
  --subnet-id $AWS_SUBNET_PUBLIC_ID \
  --query 'Instances[0].InstanceId' \
  --output text)

ssh to your VM:

AWS_INSTANCE_PUBLIC_IP=$(aws ec2 describe-instances \
  --query "Reservations[*].Instances[*].PublicIpAddress" \
  --output=text --instance-ids $AWS_INSTANCE_ID)
ssh -i vantage-key.pem ubuntu@$AWS_INSTANCE_PUBLIC_IP

Once in the VM, switch to root user:
```
sudo -i
```
Prepare the download directory for Vantage Express:
```
mkdir /opt/downloads
cd /opt/downloads
```

Install VirtualBox and 7zip:

apt update && apt-get install p7zip-full p7zip-rar virtualbox -y

Retrieve the curl command to download Vantage Express.
1. Go to Vantage Expess download page (registration required).
2. Click on the latest download link, e.g. "Vantage Express 17.20". You will see a license agreement popup. Don’t accept the license yet.
3. Open the network view in your browser. For example, in Chrome press F12 and navigate to Network tab:
4. Accept the license by clicking on I Agree button and cancel the download.
5. In the network view, find the last request that starts with VantageExpress. Right click on it and select Copy → Copy as cURL:

Head back to the ssh session and download Vantage Express by pasting the curl command. Add -o ve.7z to the command to save the download to file named ve.7z. You can remove all the HTTP headers, e.g.:

curl -o ve.7z 'http://d289lrf5tw1zls.cloudfront.net/database/teradata-express/VantageExpress17.20_Sles12_202108300444.7z?Expires=1638719978&Signature=GKBkNvery_long_signature__&Key-Pair-Id=********************'

Unzip the downloaded file. It will take several minutes:
```
7z x ve.7z
```

Start the VM in VirtualBox. The command will return immediately but the VM init process will take several minutes:

export VM_IMAGE_DIR="/opt/downloads/VantageExpress17.20_Sles12"
DEFAULT_VM_NAME="vantage-express"
VM_NAME="${VM_NAME:-$DEFAULT_VM_NAME}"
vboxmanage createvm --name "$VM_NAME" --register --ostype openSUSE_64
vboxmanage modifyvm "$VM_NAME" --ioapic on --memory 6000 --vram 128 --nic1 nat --cpus 4
vboxmanage storagectl "$VM_NAME" --name "SATA Controller" --add sata --controller IntelAhci
vboxmanage storageattach "$VM_NAME" --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium  "$(find $VM_IMAGE_DIR -name '*disk1*')"
vboxmanage storageattach "$VM_NAME" --storagectl "SATA Controller" --port 1 --device 0 --type hdd --medium  "$(find $VM_IMAGE_DIR -name '*disk2*')"
vboxmanage storageattach "$VM_NAME" --storagectl "SATA Controller" --port 2 --device 0 --type hdd --medium  "$(find $VM_IMAGE_DIR -name '*disk3*')"
vboxmanage modifyvm "$VM_NAME" --natpf1 "tdssh,tcp,,4422,,22"
vboxmanage modifyvm "$VM_NAME" --natpf1 "tddb,tcp,,1025,,1025"
vboxmanage startvm "$VM_NAME" --type headless
vboxmanage controlvm "$VM_NAME" keyboardputscancode 1c 1c

ssh to Vantage Express VM. Use root as password:
```
ssh -p 4422 root@localhost
```
Validate that the DB is up:
```
pdestate -a
```
If the command returns PDE state is RUN/STARTED. DBS state is 5: Logons are enabled - The system is quiescent, it means that Vantage Express has started. If the status is different, repeat pdestate -a till you get the correct status.
Once Vantage Express is up and running, start bteq client command line client. BTEQ (pronounced “bee-teek”) is a general-purpose, command-based client tool used to submit SQL queries to a Teradata Database.
```
bteq
```
Once in bteq, connect to your Vantage Express instance. When asked for the password, enter dbc:
```
.logon localhost/dbc
```

Run sample queries

Using dbc user, we will create a new database called HR. Copy/paste this query and run press Enter:
```
CREATE DATABASE HR
AS PERMANENT = 60e6, -- 60MB
    SPOOL = 120e6; -- 120MB
```
x

Were you able to run the query?

Let’s create a sample table and insert some data and query it. We will first create a table to hold employee information:

CREATE SET TABLE HR.Employees (
   GlobalID INTEGER,
   FirstName VARCHAR(30),
   LastName VARCHAR(30),
   DateOfBirth DATE FORMAT 'YYYY-MM-DD',
   JoinedDate DATE FORMAT 'YYYY-MM-DD',
   DepartmentCode BYTEINT
)
UNIQUE PRIMARY INDEX ( GlobalID );

Now, let’s insert a record:

INSERT INTO HR.Employees (
   GlobalID,
   FirstName,
   LastName,
   DateOfBirth,
   JoinedDate,
   DepartmentCode
)
VALUES (
   101,
   'Adam',
   'Tworkowski',
   '1980-01-05',
   '2004-08-01',
   01
);

Finally, let’s see if we can retrieve the data:

SELECT * FROM HR.Employees;

You should get the following results:

GlobalID  FirstName  LastName   DateOfBirth  JoinedDate  DepartmentCode
--------  ---------  ---------- -----------  ----------  --------------
     101  Adam       Tworkowski  1980-01-05  2004-08-01               1

Optional setup

If you intend to stop and start the VM, you may want to add Vantage Express to autostart. ssh to your VM and run the following commands:

sudo -i

cat <<EOF >> /etc/default/virtualbox
VBOXAUTOSTART_DB=/etc/vbox
VBOXAUTOSTART_CONFIG=/etc/vbox/autostart.cfg
EOF

cat <<EOF > /etc/systemd/system/vantage-express.service
[Unit]
Description=vm1
After=network.target virtualbox.service
Before=runlevel2.target shutdown.target
[Service]
User=root
Group=root
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
ExecStart=/usr/bin/VBoxManage startvm vantage-express --type headless
ExecStop=/usr/bin/VBoxManage controlvm vantage-express savestate
[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable vantage-express
systemctl start vantage-express

If you would like to connect to Vantage Express from the Internet, you will need to open up firewall holes to your VM. You should also change the default password to dbc user:
1. To change the password for dbc user go to your VM and start bteq:
  bteq
2. Login to your database using dbc as username and password:
  .logon localhost/dbc
3. Change the password for dbc user:
  MODIFY USER dbc AS PASSWORD = new_password;
4. You can now open up port 1025 to the internet:
  aws ec2 authorize-security-group-ingress \ --group-id $AWS_CUSTOM_SECURITY_GROUP_ID \ --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 1025, "ToPort": 1025, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "Allow Teradata port"}]}]'

Cleanup

To stop incurring charges, delete all the resources:

# Delete the VM
aws ec2 terminate-instances --instance-ids $AWS_INSTANCE_ID --output text

# Wait for the VM to terminate

# Delete custom security group
aws ec2 delete-security-group \
  --group-id $AWS_CUSTOM_SECURITY_GROUP_ID

# Delete internet gateway
aws ec2 detach-internet-gateway \
  --internet-gateway-id $AWS_INTERNET_GATEWAY_ID \
  --vpc-id $AWS_VPC_ID &&
  aws ec2 delete-internet-gateway \
  --internet-gateway-id $AWS_INTERNET_GATEWAY_ID

# Delete the custom route table
aws ec2 disassociate-route-table \
  --association-id $AWS_ROUTE_TABLE_ASSOID &&
  aws ec2 delete-route-table \
  --route-table-id $AWS_CUSTOM_ROUTE_TABLE_ID

# Delete the public subnet
aws ec2 delete-subnet \
  --subnet-id $AWS_SUBNET_PUBLIC_ID

# Delete the vpc
aws ec2 delete-vpc \
  --vpc-id $AWS_VPC_ID

Next steps

Query data stored in object storage