David Regalado Profile picture
May 7 โ€ข 11 tweets โ€ข 12 min read
Without effective testing, there's no way to know if your database has been migrated correctly. There are many things you need to verify.

๐Ÿงต1/10

#databases #databasemigration #dataengineering #sql #googlecloud
1. Was the database schema migrated correctly?
2. Has all the data been migrated?
3. How about user logins?
4. Can all of the users still connect and can users only access the data they're permitted to access?

๐Ÿงต2/10

#databases #databasemigration #dataengineering #sql
There are basically three categories of testing that needs to be considered; structural, functional and non-functional.

๐Ÿงต3/10

#databases #databasemigration #dataengineering #sql #googlecloud
Structured testing. Test to ensure that:

๐Ÿ‘‰ All tables have been migrated.
๐Ÿ‘‰ Fields, data types, lengths, and constraints were migrated as expected.
๐Ÿ‘‰ All indexes have been created.
๐Ÿ‘‰ Primary-foreign key relationships have been created.

๐Ÿงต4/10

#dataengineering #sql
Structured testing. Test to ensure that:

๐Ÿ‘‰ Primary and foreign keys match correctly in related tables.
๐Ÿ‘‰ All stored procedures and triggers were migrated.
๐Ÿ‘‰ User logins were migrated successfully.

๐Ÿงต5/10

#databases #databasemigration #dataengineering #sql #googlecloud
Functional testing. Test to ensure that:

๐Ÿ‘‰ All data was migrated correctly.
๐Ÿ‘‰ Stored procedures and triggers work as expected.
๐Ÿ‘‰ User logins wok as expected.

๐Ÿงต6/10

#databases #databasemigration #dataengineering #sql #googlecloud
Functional testing. Test to ensure that:

๐Ÿ‘‰ User can only perform operations and access data as permitted.
๐Ÿ‘‰ Querys and views continue to return expected results.

๐Ÿงต7/10

#databases #databasemigration #dataengineering #sql #googlecloud
Non-functional testing. Types of non-functional tests include:

๐Ÿ‘‰ Performance testing to ensure that queries and transactions complete quickly enough to meet user expectations.

๐Ÿงต8/10

#databases #databasemigration #dataengineering #sql #googlecloud
Non-functional testing. Types of non-functional tests include:

๐Ÿ‘‰ Load testing to ensure that the application performs at peak demand.
๐Ÿ‘‰ Stress testing to find the load at which the database breaks.

๐Ÿงต9/10

#databases #databasemigration #dataengineering #sql #googlecloud
Do you know another category of testing that needs to be considered?

--
๐Ÿ‘จโ€๐Ÿ’ป

๐Ÿ‘‰ I tweet about my data-journey and some tips and resources along the way. Follow me for more.

๐Ÿงต10/10

#databases #databasemigration #dataengineering #sql #googlecloud

โ€ข โ€ข โ€ข

Missing some Tweet in this thread? You can try to force a refresh
ใ€€

Keep Current with David Regalado

David Regalado Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @thecodemancer_

Mar 27
Can you imagine serverless Spark + BigQuery together? ๐Ÿคฏ

Forget about managing clusters and tuning infrastructure if your job is to focus on create business value.

๐Ÿ‘‡

๐Ÿงต1/6

#googlecloud #bigquery #spark #dataengineering
Why Serverless Spark?

๐Ÿ’ก Developers can focus on code and logic. They do not need to manage clusters or tune infrastructure. They submit #Spark jobs from their interface of choice, and processing is auto-scaled to match the needs of the job.

๐Ÿงต2/6

#googlecloud #bigquery #gcp
๐Ÿ’ก Data engineering teams do not need to manage and monitor infrastructure for their end users. They are freed up to work on higher value #dataengineering functions.

๐Ÿ’ก Pay only for the job duration, vs paying for infrastructure time.

๐Ÿงต3/6

#googlecloud #bigquery #spark
Read 7 tweets
Mar 18
Bored employees are super disengaged, prone to conflict and suffer burnout at a higher rate.

Harvard Business Review found employees who are learning at work experience less anxiety and stress. They are also more ethical than bored workers on autopilot.

๐Ÿงต1/7

#manager #career
All employees deserve to learn, bored, engaged or somewhere in between. Itโ€™s your job as a manager to make that happen. Here are three ideas to create a learning strategy and become a manager whose people wonโ€™t leave:

๐Ÿงต2/7

#manager #career
1. Plan to learn something new every week.

Invite your employees to schedule 10 minutes on their calendars every day to learn something new. Then, weekly, ask your employees about what theyโ€™ve learned.

๐Ÿงต3/7

#manager #career #learning
Read 7 tweets
Mar 16
Is Infrastructure as SQL (IaSQL) a thing? ๐Ÿค”

๐Ÿงต1

#dataengineering #sql #IaSQL #cloud
๐—›๐—ผ๐˜„ ๐—œ๐—ฎ๐—ฆ๐—ค๐—Ÿ ๐˜„๐—ผ๐—ฟ๐—ธ๐˜€

IaSQL is an open-source SaaS that models cloud infrastructure as data by maintaining a 2-way connection between a AWS account and a hosted PostgreSQL database. โ˜

๐Ÿงต2
๐—ช๐—ต๐˜† ๐—œ๐—ฎ๐—ฆ๐—ค๐—Ÿ?

๐Ÿ‘‰ ๐˜๐˜ฎ๐˜ฑ๐˜ฐ๐˜ณ๐˜ต ๐˜ฆ๐˜น๐˜ช๐˜ด๐˜ต๐˜ช๐˜ฏ๐˜จ ๐˜ค๐˜ญ๐˜ฐ๐˜ถ๐˜ฅ ๐˜ช๐˜ฏ๐˜ง๐˜ณ๐˜ข๐˜ด๐˜ต๐˜ณ๐˜ถ๐˜ค๐˜ต๐˜ถ๐˜ณ๐˜ฆ

Connect an AWS account to a hosted IaSQL DB to automatically backfill the database with your existing cloud resources. No need to redefine or reconcile existing infrastructure.

๐Ÿงต3
Read 6 tweets
Feb 26
The evolution of data processing frameworks.

Knowing how these frameworks have evolved can help you understand the typical problems that arise, and how they're addressed.

As the Internet grew, Google invented new data processing methods.

๐Ÿงต

#GCP #google @google @googlecloud
In 2002, Google created GFS, or the Google File System to handle sharding and storing petabytes of data at scale.

GFS is a foundation for cloud storage, and also for what would become BitQuery managed storage.

๐Ÿงต

#GCP #google @google @googlecloud
One of the next challenges was to figure out how to index the exploding volume of content on the Web.

To solve this, in 2004 @Google invented a new style of data processing (MapReduce) to manage large scale data processing across large clusters of commodity servers.

๐Ÿงต

#GCP
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(