I joined @sheplusplus because I wanted to be a part of a community of ambitious women and help other ambitious women make strides in their careers (namely tech). But quarterly/annual dues were a large part of the reason behind my decision not to join a sorority in college. (2/7)
When the financial barrier to entry is high for minorities in the field, the only minorities who can join the group have to be extra privileged. To actually increase diversity in the field, we need to make efforts to include those who can't afford large membership dues. (3/7)
I learned this the hard way. A she++ program I ran, the #include Fellowship, offered a free trip to Bay Area for select high school students who demonstrated an interest in CS education. We ended up indirectly selecting HS kids who spent lots of $$ hosting hackathons, etc. (4/7)
Majority of the kids we selected were from large cities (Bay Area, NYC, Seattle) and had coding experience. I feel like I could have made a lot more of an impact if we had targeted HS kids in more rural areas and not required them to have demonstrated CS education interest. (5/7)
Bottom line is, it's super important to empower women, but it's even more important to be as inclusive as possible for all identifying women. I want to see more initiatives (especially in tech) that reach women from lower socioeconomic backgrounds, more rural areas, etc. (6/7)
@the_wing could drastically lower membership costs for individuals and (maybe) make $$ by adopting an accelerator business model. I know it's hard, but I'd love to hear others' thoughts on sustainable economics behind coworking spaces for minorities in a field! (7/7)
• • •
Missing some Tweet in this thread? You can try to
force a refresh
recently been studying prompt engineering through a human-centered (developer-centered) lens. here are some fun tips i’ve learned that don’t involve acronyms or complex words
if you don’t exactly specify the structure you want the response to take on, down to the headers or parentheses or valid attributes, the response structure may vary between LLM calls / it is not amenable to production
play around with the simplest prompt you can think of & run it a bunch of times on different inputs to build intuition for how LLMs “behave” for your task. then start adding instructions to your prompt in the form of rules, e.g., “do not do X”
thinking about how, in the last year, > 5 ML engineers have told me, unprompted, that they want to do less ML & more software engineering. not because it’s more lucrative to build ML platforms & devtools, but because models can be too unpredictable & make for a stressful job
imo the biggest disconnect between ML-related research & production is that researchers aren’t aware of the human-centric efforts required to sustain ML performance. It feels great to prototype a good model, but on-calls battling unexpected failures chip away at this success
imagine that your career & promos are not about demonstrating good performance for a fixed dataset, but about how quickly on average you are able to respond to every issue some stakeholder has with some prediction. it is just not a sustainable career IMO
Been working on LLMs in production lately. Here is an initial thoughtdump on LLMOps trends I’ve observed, compared/contrasted with their MLOps counterparts (no, this thread was not written by chat gpt)
1) Experimentation is tangibly more expensive (and slower) in LLMOps. These APIs are not cheap, nor is it really feasible to experiment w/ smaller/cheaper models and expect behaviors to stay consistent when calling bigger models
1.5) we know from MLOps research that high experimentation velocity is crucial for putting and keeping pipelines in prod. A fast way is to collect a few examples, load up a notebook, try out a heck of a lot of different prompts—calling for prompt versioning & management systems
IMO the chatgpt discourse exposed just about how many people believe writing and communication is only about adhering to some sentence/paragraph structure
I’ve been nervous for some time now, not because I think AI is going to automate away writing-heavy jobs, but because the act of writing has been increasingly commoditized to where I’m not sure whether people know how to tell good writing from bad writing. Useful from useless.
In my field, sometimes it feels like blog posts (that regurgitate useless commentary or make baseless forecasts about the future) are more celebrated/impactful than tooling and thought. Often such articles are written in the vein of PR or branding
I want to talk about my data validation for ML journey, and where I’m at now. I have been thinking about this for 6 ish years. It starts with me as an intern at FB. The task was to classify FB profiles with some type (e.g., politician, celebrity). I collected training data,
Split it into train/val/test, iterated on the feature set a bit, and eventually got a good test accuracy. Then I “productionized” it, i.e., put it in a dataswarm pipeline (precursor to Airflow afaik). Then I went back to school before the pipeline ran more than once.
Midway through my intro DB course I realized that all the pipeline was doing was generating new training data and model versions every week. No new labels. So the pipeline made no sense. But whatever, I got into ML research and probably would never do ML in industry again.
Our understanding of MLOps is limited to a fragmented landscape of thought pieces, startup landing pages, & press releases. So we did interview study of ML engineers to understand common practices & challenges across organizations & applications: arxiv.org/abs/2209.09125
The paper is a must-read for anyone trying to do ML in production. Want us to give a talk to your group/org? Email shreyashankar@berkeley.edu. You can read the paper for the war stories & insights, so I’ll do a “behind the scenes” & “fave quotes” in this thread instead.
Behind-the-scenes: another school invited my advisor to contribute to a repo of MLOps resources. We contributed what we could, but felt oddly disappointed by the little evidence we could point to for support.