Thread by @WeAreRLadies

I'm sitting down to start a task in R.

I don't entirely know how to complete it.

I'm going to try to document my process in this thread in real time.

The rubric for my students' projects includes "Consistency of effort" which is partly assessed by examining their github commit history over the weeks since they started.

The task: For each student, given their github repo urls, I need to get their commit history: commit datetimes, sizes and perhaps commit messages.

I already know:
* if I can do it for one student I can do it for all students with purrr::map().
* Github has an API, and I'm hoping the API includes a way to query commits.
* I'm pretty sure I've come across a R package that accesses this API, but I don't remember its name.

I've opened my existing RStudio project for this class, opened a new Rmarkdown file, copied my task in, included an actual project url for one student, saved the file.

I do a web search for "github api r", scan for github and cran results, and come across github.com/cscheid/rgithub and github.com/r-lib/gh

I scan both github READMEs and see on rgithub "If you're a user of this library, consider switching to gh or ghapi3", I rule out rgithub.

I take a quick look at ghapi3, see it is WIP and the build is currently failing...decide to focus of gh

Scan gh github page. See it has recent commits and is on cran...good signs. Skim README and see "Use the gh() function to access all API endpoints." and links to github docs...another good sign for my task.

Skim developer.github.com/v3/ and figure out I'm interested in Repositories, and "The Repo Commits API supports listing, viewing, and comparing commits in a repository." Perfect.

OK, how does the gh package work? I now work through the README, starting with installing the package (although I choose the CRAN version not the dev one), then running the examples one by one in my local RStudio.

Impressed that my_repos actually lists my repos (not @GaborCsardi s), giving me hope that I have some existing environment variables set up that help with identification and authorization.

@GaborCsardi Stop doing examples on README when I hit "POST, PATCH, PUT and DELETE requests" because I'm pretty sure I won't have to do this.

@GaborCsardi My student's repos are private, can I see them? I edit the README example to list my *private* repos, and do a little fist pump when I see some of my students repos there!

@GaborCsardi ...but then get puzzled about why I can't see them all...then remember the API might have a page limit.

@GaborCsardi Spend a little time going back through current RMD and adding notes. Then spend next 12mins trying to adapt a working example to get me commits on one of my repos. Iterate between examples in gh, github API docs and trying code...

@GaborCsardi At least 8 unsuccessful calls to gh() and exploring returned objects for calls that worked I get a call to the commits endpoint to work!

@GaborCsardi Now I look at the output and realize it's a complicated list, but with one element for each commit. I extract one commit, explore it (in the RStudio viewer) and write a small function that takes one commit and returns a nice tibble with sha, dateime, committer, and message.

@GaborCsardi I confirm function works for one commit, then use map_dfr() to do it for list of commits, and celebrate the nice tibble it returns.

@GaborCsardi Now I try to repeat this for a specific students repo. Keep getting 404 errors and can't decide if I'm specifying the endpoint wrong, or I'm running into github permissions problems

@GaborCsardi Read ?gh(), head to ?gh_whoami() and confirm I'm the user I expect and seem to have the right scope ("repo" - "Grants read/write access to ... and private repositories and organizations.", so now thinking I'm messing up the endpoint

@GaborCsardi Careful review of working code and comparing to non-working code reveals typo...groan... "/user" should have been "/repo"...it works for one student!

@GaborCsardi Just as I realise I've been at this for at least an hour, colleague knocks on door for coffee run. I delay for 2mins, quickly make notes on current progress in Rmd ("working for one student, need to extend to many students"). Post here. Will come back to task later. ☕ time

@GaborCsardi Coffee turned into student's dropping by with questions, and now I need to head to a training...task on pause.

@GaborCsardi 🏋️‍♀️🚴‍♀️🚿💻🍲📺💤🚿☕🚴‍♀️👩‍🏫 later...I finally get back to the task...have to reopen project, rerun all existing code: it still works! Try plotting one students commit history and realise I haven't parsed commit times as datetimes yet, add lubridate::ymd_hms() + lubridate::with_tz()

@GaborCsardi Grab two more student repo URLS and put them in a vector, do a stringr::str_replace() to just keep repo name. For each repo, get commits with purrr::map() and a recipe I have getting one students commits.

For each students' commits, get commit summary using function I wrote earlier. Celebrate that I now have commit summaries for three students in a list.

Realise I should have put everything in a tibble. Refactor to put repo names in a tibble, rewrite map steps to use mutate inside tibble, unnest. Now I have a tibble with one row per commit, with a repo_name columns keeping track of which repo the commit was made to.

Make a plot...confirm things look OK. Tweak plot, anonymize repo names, save plot.

Decide I'm done for now. Actual projects aren't due until Sun, but I'm convinced that I have enough that I can repeat this with 28 repos next week.

END OF THREAD.