Discover and read the best of Twitter Threads about #spark

Most recents (19)

Learning data science on your own is tough...

...(ahem, it took me 6 years)

So here's some help.

5 Free Books to Cut Your Time In HALF.

Let's go! 🧵

#datascience #rstats #R Image
1. Mastering #Spark with #R

This book solves an important problem- what happens when your data gets too big?

For example, analyzing 100,000,000 time series.

You can do it in R with the tools covered in this book.

Website: therinspark.com Image
2. Geocomputation with #R

Interested in #Geospatial Analysis?

This book is my go-to resource for all things geospatial.

This book covers:
-Making Maps
-Working with Spatial Data
-Applications (Transportation, Geomarketing)

Website: r.geocompx.org Image
Read 8 tweets
[1/🧵] Is this you? 🤨

• Holder of #XRP since December 2020
• Has never heard of $FLR before (Hello, #Spark?)
• Has no idea what the fuss is all about
• Slept under a rock for two years

Allow me to bring you up to speed w/ this crash course on @FlareNetworks and $FLR. 🧵👇 Source: https://timestabloid.com/flare-ceo-updates-xrp-holde
[2/20] If you've been following this topic, you'll recall that there was once a #XRPL snapshot:

• 12th December 2020, 00:00 UTC
#XRP Ledger index 60,155,580
. . . Source: https://xrpscan.com/ledger/60155580
[3/20] . . .

You may recall that you had to claim your #Spark coins by providing your #EVM-based $FLR address in the message key field of your #XRPL account.

→ claim period expired on #XRPLedger index 64,204,315
→ Message could be changed until 11th June 2021, 23:59 UTC
. . . Source: https://xrpscan.com/ledger/64204315
Read 22 tweets
Free SQL Resources for beginners!

Spend $0 and earn yourself some SQL knowledge with these sites.

Post Credit - Travis Tang Sir

🧵👇
1️⃣ 𝙆𝙝𝙖𝙣 𝘼𝙘𝙖𝙙𝙚𝙢𝙮's Intro to SQL: Querying and Managing Data comes with video lessons and interactive coding exercises.
(lnkd.in/g5y_Pfvf)
2️⃣ 𝘾𝙤𝙙𝙚𝙖𝙘𝙖𝙙𝙚𝙢𝙮 provides a treasure trove of text tutorials for beginners and intermediates alike. (lnkd.in/gZ5_uYjC)
Read 6 tweets
Today we are glad to announce these exciting features! 🎉

* Improved StatsForecast class
* Selection of best-performing models
* New documentation
* Plotting functionalities

Stay tuned for an exciting polemic comparison tomorrow!

#reinventforecasting
StatsForecast class:

With the new StatsForecast class, you can fit many models for many time series in less than 15 lines of code

You can include @dask_dev, @raydistributed, and #spark by changing just the backend parameter Image
Models: StatsForecast 27 models.

They can be included and imported in a few lines of code. Image
Read 7 tweets
Today at @nixtlainc: fast and interpretable forecasting for multiple seasonalities. ⏲️🔎

This is exciting news for data scientists in the Python ecosystem. 🧵
#reinventforecasting
Time series are the operational DNA of the world. 🧬

Most business data (sales, demand, electricity load, sensors) is stored in tabular format with time indexes.

Most forecasting models in Python are only suited for single seasonal data. ⚠️
#reinventforecasting
However, many time series have multiple seasonalities.

Multiple seasonality is traditionally present in data sampled at a low frequency. 💡

For example, hourly electricity data exhibits daily and weekly seasonality. 🎄🕎❄️

#reinventforecasting
Read 7 tweets
@nixtlainc started last year as a side project

Today we reached 1 million downloads🎉

Our goal is to shake the time series industry and make state-of-the-art algorithms available for everyone 🔥

This is how we got there

🧵1/11

#reinventforecasting
@nixtlainc's ecosystem consists (now) of 5 #python libraries

Focus: speed, scalability, and accuracy 🚀

Some features:
* @scikit_learn syntax
* Native support for #spark, @raydistributed, and @dask_dev
* Models! Eg #arima, #LightGBM, #NeuralNetworks, #transformers 🤖

2/11 Image
We are really proud of the open-source adoption

Repos from Amazon, Mozilla, and DataBricks use us as dependencies 🏄‍♂️

We have contributions from people working at H20, Microsoft, Google, Facebook, SalesForce, Oracle, Shopify, AT&T, Blueyonder, Stanford, MIT and UCL 🙏

3/11
Read 12 tweets
Ever wondered how to join big data in R?

A thread on using Spark in R👇🧵

#rstats #spark #datascience
This is thread # 3 in a series exploring using Spark in R with the sparklyr package. You can find the others here:

# 1:


# 2:

1/17
Here's what you'll learn reading this thread:

1. How a regular left join works.

2. The sort-merge-join algorithm (the default way to join big data with Spark and sparklyr).

3. Broadcast joins.

4. Salted joins.

All directly in R. Let's go.

2/17
Read 18 tweets
Ever wondered how to manipulate big data with R?

A thread on using Spark in R👇🧵

#rstats #spark #datascience
Big data is any data which is too large to be loaded into memory all in one go. Ever tried to read a large dataset into R and everything fell over or slowed waaaaaay down? This is where Spark comes in.

1/18
Spark is an open source tool for processing big data. It splits data into partitions for processing to overcome RAM limitations and writes to disk when needed. It is complicated but Spark handles most of the difficult parts. And you can work with Spark directly from R.

2/18
Read 19 tweets
Can you imagine serverless Spark + BigQuery together? 🤯

Forget about managing clusters and tuning infrastructure if your job is to focus on create business value.

👇

🧵1/6

#googlecloud #bigquery #spark #dataengineering
Why Serverless Spark?

💡 Developers can focus on code and logic. They do not need to manage clusters or tune infrastructure. They submit #Spark jobs from their interface of choice, and processing is auto-scaled to match the needs of the job.

🧵2/6

#googlecloud #bigquery #gcp
💡 Data engineering teams do not need to manage and monitor infrastructure for their end users. They are freed up to work on higher value #dataengineering functions.

💡 Pay only for the job duration, vs paying for infrastructure time.

🧵3/6

#googlecloud #bigquery #spark
Read 7 tweets
🧵1. #IPERN officially launched on #StPatricksDay 2021 so it's our 1st birthday today!🎂🚑☘ It's been quite a year - we've hosted 10 events featuring 49 speakers (31% International 🌍 65% Female 🚺). Presented at 3 conferences, secured 3 grants and launched 5 SIG! #IrishEMS
2. How it all began… we announced ourselves to the Twitter world on Feb 1st '21. In Ireland this is traditionally the start of Spring (#Imbolc) so a time for new beginnings. It’s also St. Brigid‘s Day our matron saint who Is renowned as a healer #IrishEMS tinyurl.com/2p8w2zwt
3. Our official #IPERN launch on #StPatricksDay 2021 featured a fantastic array of Irish researchers presenting "elevator pitches" of their work and also included best wishes from international friends & colleagues! tinyurl.com/2p92at99 #IrishEMS
Read 34 tweets
I want to start a new thread story. Not sure how long it will be and I may use different prompts: #vss365 #vssmagic #vssnature etc etc ... I love reading about legends and folk lore, and in my fantasy writing I often create my own myths. So this will be just a fun exercise...
According to #lore &myths of old,griffins nestled on these cliffs long ago. They swooped down on the unsuspecting;carried them off never to be seen again.
Griffins were not elemental creatures,but air & earth did have dominion over them; their only enemy -fire.
#vss365 #vssmagic
"I helped your troupe avoid #shadow and #gore," said the dark wizard simply, implying that he saved us from the mythical griffins. I gaped at him. "And now for what you promised."

"No," I was careful not to get trapped in words, "I said: after you summon her."
#vss365 #vssmagic
Read 132 tweets
Discordで教えてもらったんだけど、新しいMEW使ってる場合は24のシードフレーズでメタマスク上に復元できるので、インポートとかせずに直で復元すれば無問題のようです。
つまり、いずれBifrost wallet使いたくて、シードフレーズがあるMEWのETHアドレスで #FLR のオプトイン申請した人は、メタマスクのETHアドレスで申請し直さなくてもOK。 #XRP #FLR #Spark #FlareNetworks
Bifrost wallet は接続にシードフレーズが必要で、秘密鍵では接続できないから、シードフレーズがない旧版MEWのETHアドレスとは接続できないよ〜ということです。(多分)
Read 4 tweets
Assuming you're not using aliases, a plain text search in @obsdmd for "[[page]]" within quotations pulls up the backlinks for that page. Searching for ".png" will pull up all of your pngs. There's a lot of flexibility that comes from searching strings when everything is in markup
Here I have an embedded query to find all my notes directly dealing with DSLs (domain-specific languages). I get this by searching for file names that have DSL in them. In Roam I might have had [[A [[DSL]] speeds up the author and gives them the ability to write from anywhere]] ImageImage
Here I'm trying out using namespacing in a @dendronhq style. I have an embedded search looking for pages that have Clojure and function in the title. It's not officially supported in Obsidian, but with searches embedded on pages, I'm able to do it quite flexibly ImageImage
Read 10 tweets
How can I start a career in #risk management in the financial services industry?
@GARP_Risk @PRMIA @RIMSorg @actuarynews @SOActuaries @CQFInstitute @CFAinstitute
I believe they are many degrees that can be a good value for money, provided you know where you would like to work in the long run.
In the developing world markets, employers still don't appreciate highly technical education in risk, actuarial sciences, financial engineering, quantitative economics, machine learning, etc.

For them, an #MBA is a be-all and an end-all troubleshooter.
Read 22 tweets
1/365 - Starting this thread with one of the first Motorcycles to be built in India. The @royalenfield #Bullet350. A Motorcycle which was initially built for the Indian Army but with time found patronage amongst varying genre of people achieving a cult status over the decades. ImageImageImageImage
2/365 - After selling imported Vespas for a while, Bajaj Auto started manufacturing the #Bajaj150 under license in 1959. This was one of the first Scooters in India and certainly changed the 2-Wheeler scene as it was initially available at a significant premium in the market. ImageImageImageImage
3/365 Automobile Products of India (API) was the 1st Scooter maker in India and manufactured the #Lambretta Li 150 Series 2 under license from Innocenti at Bombay in the 1960's. This scooter was very popular and made appearances in a few famous Bollywood movies too. #BikerLife ImageImageImageImage
Read 73 tweets
🚨¡Atención! Bitso participará en el snapshot 📸 de #XRP para la distribución del token #Spark
2/ ☝️Este snapshot será tomado el 11 de diciembre de 2020 a las 18:00 hora de la Ciudad de México, 21:00 hora de Argentina, (00:00 UTC). ⏲️
3/ Es importante que sepas que un snapshot 📸 es cuando se registran los balances de todas las wallets en un momento específico del tiempo. En este caso, se refiere a una “fotografía” de los balances XRP en toda la red de XRP.
Read 10 tweets
#100daysofCPD is where @UoB_HEFi will be beaming a spotlight onto #teaching and supporting #learning in #HigherEducation #CPD opportunities especially for #Beacon #HEAfellowship applicants & @uob_pgche participants during #COVID19 times & beyond Image
#100daysofCPD: #Day1 is the @UoB_HEFi #Learning and #Teaching Gateway. Full of self-paced online courses on a range of #technology and teaching topics. #BestPractice #CPD Go to birmingham.ac.uk/university/hef… Image
#100daysofCPD #Day2: @matt_tel_brum shares this Resource - How to Keep Uninvited Guests Out of Your Zoom Event containing all you need to know about Zoom’s #settings when using for #teaching, #meetings or #socializing during #COVID19 & beyond blog.zoom.us/wordpress/2020…
Read 106 tweets
So, my thread on how to choose databases for your web apps starts in 20 minutes time. Hope y'all are ready ?However, this time i would love it to be an interactive session where we can learn from each other about database preferences for specific web apps and the use case
#thread
This is what we want to talk about today. Take a really good look at the image below. Study it well. Also notice the words at the 3 vertexes of the triangle. CONSISTENCY, AVAILABILITY & PARTITION-TOLERANCE (Big English - but fear not, we shall demystify all and it'll all be easy)
If we also look at the image in the previous tweet on this thread we can see some popular databases placed at certain sides of the triangle. It is important to note also the positioning of some of a particular point on each side. We shall discuss this in more detail soon. #thread
Read 48 tweets
@AdobeSpark #Feedback. Pinch-in & Pinch-out are not very effective interaction to resize the text boxes precisely in Adobe Spark Android Mobile App. They are good for Zoom-in & Zoom-out from Maps because it don't require precision.
Sometimes when I try to left/right align two text boxes, suppose Title and Content. The auto snap helps me to properly align them but as I lift my finger, the text box slightly displace. That is irritating. I need a tool to fine-tune the movement. This interaction is
for Android Mobile App.

I am not sure that Adobe Spark primarily focusing on Print Media or Social Graphic. As I use Spark mostly for publishing Social Graphic thus I think "Popular" and "Stand out in Social" must be swapped in this page spark.adobe.com/sp/projects.
Read 7 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!