Discover and read the best of Twitter Threads about #googlecloud

Most recents (11)

Did you notice this?

Some #GoogleCloud professional certificates on Coursera have off-platform certification exams. For a limited time, you can get a discount voucher for 20% off the cost of the exam.

This is a 🧵of links to those programs.

🧵[1/x]

@coursera @GoogleCloudTech
Google Cloud Digital Leader Training Professional Certificate lnkd.in/efnhSb57

🧵[2/x] Image
Preparing for Google Cloud Certification: Cloud Data Engineer Professional Certificate lnkd.in/eiwqGYxt

🧵[3/x] Image
Read 7 tweets
📣Data Engineering Projects for Beginners 2022

👇🧵[1/x]

#dataengineering #python #Docker #developers #aws #GoogleCloud #apacheairflow
Tracking your Uber Rides and Uber Eats expenses through a data engineering process

Technologies and skills:
Python, Docker, Apache Airflow, AWS Redshift, Power BI, data modelling, Task schedulling, ETL and ELT processes, Data warehousing, Cloud

🧵[2/x]

github.com/Wittline/uber-…
Scheduling Big Data Workloads and Data Pipelines in the Cloud with pyDag

Technologies and skills:
Python, Docker, Big Data, Cloud, Google Cloud, Redis, DAG, Parallel Processing, Apache Spark

🧵[3/x]

github.com/Wittline/pyDag
Read 15 tweets
☁ You will likely encounter pushback when moving to the cloud. Moving to something new may seem risky and unnecessary to the developers. This requires a cultural shift.

💎 Here are some tips on how to tackle this problem.

#cloud #googlecloud #azure #aws
1. Sync with cross-functional teams early and often. Train them so they understand the benefits of the cloud and are comfortable and knowledgeable using it.
2. Help teams understand the benefits, the project's processes, the desired goals and outcomes.
Read 6 tweets
Without effective testing, there's no way to know if your database has been migrated correctly. There are many things you need to verify.

🧵1/10

#databases #databasemigration #dataengineering #sql #googlecloud
1. Was the database schema migrated correctly?
2. Has all the data been migrated?
3. How about user logins?
4. Can all of the users still connect and can users only access the data they're permitted to access?

🧵2/10

#databases #databasemigration #dataengineering #sql
There are basically three categories of testing that needs to be considered; structural, functional and non-functional.

🧵3/10

#databases #databasemigration #dataengineering #sql #googlecloud
Read 11 tweets
Can you imagine serverless Spark + BigQuery together? 🤯

Forget about managing clusters and tuning infrastructure if your job is to focus on create business value.

👇

🧵1/6

#googlecloud #bigquery #spark #dataengineering
Why Serverless Spark?

💡 Developers can focus on code and logic. They do not need to manage clusters or tune infrastructure. They submit #Spark jobs from their interface of choice, and processing is auto-scaled to match the needs of the job.

🧵2/6

#googlecloud #bigquery #gcp
💡 Data engineering teams do not need to manage and monitor infrastructure for their end users. They are freed up to work on higher value #dataengineering functions.

💡 Pay only for the job duration, vs paying for infrastructure time.

🧵3/6

#googlecloud #bigquery #spark
Read 7 tweets
The evolution of data processing frameworks.

Knowing how these frameworks have evolved can help you understand the typical problems that arise, and how they're addressed.

As the Internet grew, Google invented new data processing methods.

🧵

#GCP #google @google @googlecloud
In 2002, Google created GFS, or the Google File System to handle sharding and storing petabytes of data at scale.

GFS is a foundation for cloud storage, and also for what would become BitQuery managed storage.

🧵

#GCP #google @google @googlecloud
One of the next challenges was to figure out how to index the exploding volume of content on the Web.

To solve this, in 2004 @Google invented a new style of data processing (MapReduce) to manage large scale data processing across large clusters of commodity servers.

🧵

#GCP
Read 9 tweets
Watching @NetAppINSIGHT keynote? Me too. Here are my quick thoughts. A 🧵

#NetAppINSIGHT /1
IMO, #NetApp was the traditional on-premises #Storage vendor that moved faster to the #Cloud. They created the #DataFabric vision a few years ago and are delivering value today.

#NetAppINSIGHT /2
With ONTAP services available on #AWS, #Azure and #GoogleCloud, I can compare what #NetApp is doing with #Storage to what #VMware is doing with virtual machines: providing existing customers an easier path to #Cloud migration.

#NetAppINSIGHT /3
Read 10 tweets
Day 14 #31DaysofML

🤔 How to pick the right #GoogleCloud #MachineLearning tool for your application?

Answer these questions
❓ What's your teams ML expertise?
❓ How much control/abstraction do you need?
❓ Would you like to handle the infrastructure components?

🧵 👇
@SRobTweets created this pyramid to explain the idea.
As you move up the pyramid, less ML expertise is required, and you also don’t need to worry as much about the infrastructure behind your model.

To lear more watch this video 👉

#31DaysofML 2/10
@SRobTweets If you’re using Open source ML frameworks (#TensorFlow) to build the models, you get the flexibility of moving your workloads across different development & deployment environments. But, you need to manage all the infrastructure yourself for training & serving

#31DaysofML 3/10
Read 10 tweets
Why don't they just leave #Africa alone? The #DSIAfricaProgram gives me the creeps. #IBM, #Philips, #GatesFoundation all there, ready to experiment on them. Note 'mining of text or data to discover trends' in 'Using Data Science to Fight Covid' commonfund.nih.gov/africadatasymp…
They partner with #AmazonWebServices & #GoogleCloud. Also, AI is being used for 'Biomedical Excellence' #AIBLE to create Data Design Centres- what's that? How to invent a pandemic based on fake positive cases of the non-existent virus?
Oh look, what have we got here? An International #Covid19DataResearchAlliance provided by, of course, the #GatesFoundation
Read 5 tweets
Resolved: High Google Cloud infrastructure components incident: We are investigating an issue with elevated error rates across multiple Google Cloud Platform Services #googlecloud status.cloud.google.com/incident/zall/… # ISSUE SUMMARY (All times in US/Pacific 1/
daylight time)

On Wednesday 08 April, 2020 beginning at 06:48 US/Pacific, Google Cloud Identity and Access Management (IAM) experienced significantly elevated error rates for a duration of 54 minutes. IAM is used by several Google services to manage user information, and 2/
the elevated IAM error rates resulted in degraded performance that extended beyond 54 minutes for the following Cloud services: - Google BigQuery’s streaming service experienced degraded performance for 116 minutes; - Cloud IAM’s external API returned elevated errors for 3/
Read 53 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!