---
title: "Data Visualization 2"
author: "Ryan Womack"
date: "2024-10-03"
toc:toc: true true
number-sections: true
highlight-style: pygments
output: html_document
format:
  html: 
    toc: true
    code-fold: true
    html-math-method: katex
  pdf:
    geometry: 
      - top=30mm
      - left=30mm
    fontfamily: libertinus
    colorlinks: true
    papersize: A4
  docx: default
theme: litera
include-in-header: 
  text: |
    \usepackage{fvextra}
    \DefineVerbatimEnvironment{Highlighting}{Verbatim}{breaklines,commandchars=\\\{\}}
---

```{r setup, include=FALSE}

knitr::opts_chunk$set(echo = TRUE)
knitr::opts_chunk$set(root.dir = "/home/ryan/R/data_topics/data_visualization_2/")

```

Copyright Ryan Womack, 2024. This work is licensed under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)

**Data Visualization 2**

_interactive visualizations with Shiny, including hosting a Shiny server_

# Overview

Let's start by looking at the basic documentation and gallery examples at [https://shiny.posit.co/](https://shiny.posit.co/) and [https://shiny.posit.co/r/gallery/](https://shiny.posit.co/r/gallery/). Shiny allows us to build interactive data visualizations. It now supports Python visualizations as well, although this session will use R examples.

A few examples of Shiny in action:

 - [Explore ggplot2 with Shiny](https://www.r-bloggers.com/2017/06/shiny-app-to-explore-ggplot2/)
 - [College Scorecard Data](https://shiny.ryanwomack.com/CollegeSearchTool)
 - [BTAA librarians](https://ryandata.shinyapps.io/BTAA)

## _Mastering Shiny_, by Hadley Wickham

We'll refer frequently to [Mastering Shiny](https://mastering-shiny.org/index.html) by Hadley Wickham, master R developer, which provide a complete presentation of working with Shiny beyond this introductory workshop. The open version of the text is linked above, but we also have access through Rutgers to the [O'Reilly collection](https://learning-oreilly-com.proxy.libraries.rutgers.edu/home/) which contains the [officially published version](https://learning-oreilly-com.proxy.libraries.rutgers.edu/library/view/mastering-shiny/9781492047377/), along with other useful [Shiny learning materials](https://learning-oreilly-com.proxy.libraries.rutgers.edu/videos/shiny-r/9780134863320/9780134863320-SHIR_01_00_00/) (and much more for all kinds of tech!).

# Setup

## Shiny and rsconnect

We will use the [_pak_](https://pak.r-lib.org/) package for installation as a more complete approach to package management. Replace the pkg commands with _install.packages()_ versions if you prefer.

The [_shiny_](https://tidyverse.org) package encapsulates most of what you need to run _shiny_ (of course you'd need the packages for whatever kinds of graphics you'd like to display as well). The [_rsconnect_](https://rstudio.github.io/rsconnect/) package is used to publish Shiny apps to the web, as we'll see later. Install _pak_ and _tidyverse_ if you don't already have them on your system. We will not run Python code in this session, but keep in mind that it is now possible to use Python in Shiny.

```{r install packages, eval=FALSE}

install.packages("pak", dependencies=TRUE)
library(pak)
pkg_install("shiny")
pkg_install("rsconnect")

```

Now let's load the _tidyverse_, _shiny_, and _rsconnect_.
```{r tidyverse}

library(tidyverse)
library(shiny)
library(rsconnect)

```

# Shiny basics

We'll walk through some of the [Shiny gallery examples](https://shiny.posit.co/r/gallery/) to understand the fundamentals of Shiny.

## ui and server

The _ui_ section of the code controls the user interface, and often passes variables to the server side. The ui side has a number of Shiny-specific functions to control the interface (e.g., "sliderInput") and allow us to quickly build interactivity via these building blocks.

The _server_ section of the code controls the generation of data, tables, and visualization, creating output that is passed to the ui side.  While the output generated by the server may be encapsulated by a Shiny fuction, the core output is typically generated by standard R functions (or Python if you choose).

We should use the default labeling of "input" and "output" to render our code clear and consistent, although in theory these names are arbitrarily assigned.

## Conventions and app.R

We put our Shiny files in a single directory, including supplementary files like data (unless using direct file location references). While all Shiny visualizations will have a _ui_ and _server_ component, we have two options on how to set this up:

1. Create two files, one called _ui.R_ and the other called _server.R_ with code describing the user interface and server commands respectively. We can also (optionally) use a third file called _global.R_ to take care of some settings and globally applicable commands (like loading packages).  This approach has the advantage of making the separate sides of the Shiny app easy to view and edit.

2. Create a single file, called _app.R_. This approach can be simpler to deploy, since it is obvious that everything happens in the single file, just one thing to update and upload. Inside the app.R file, we have a _ui_ section and a _server_ section. We then run the app by running _shinyApp(ui, server)_ .

Here's a very basic single file _app.R_, as an illustration.

```{r app.R}

library(shiny)

ui <- basicPage(
  plotOutput("plot1", click = "plot_click"),
  verbatimTextOutput("info")
)

server <- function(input, output) {
  output$plot1 <- renderPlot({
    plot(mtcars$wt, mtcars$mpg)
  })
  
  output$info <- renderText({
    paste0("x=", input$plot_click$x, "\ny=", input$plot_click$y)
  })
}

shinyApp(ui, server)

```

In the live workshop (and recording), we walk through creating the files in RStudio at this point.

## Reactivity

The concept of _reactivity_ is fundamental to the interactivity of the Shiny display. Reactive expressions dynamically update as inputs change.

The code wrapper _reactive({...})_ is used to turn any function into a reactive expression. Then one can call it like a function. The difference between it and a function is that the reactive expression runs once and caches its result, unless and until its inputs are updated.

We can modify the default Shiny server.R to make define its reactive as follows:

```{r reactivity}

function(input, output, session) {
  
  my_input <- reactive(input$bins)
  
  output$distPlot <- renderPlot({
    
    # generate bins based on input$bins from ui.R
    x    <- faithful[, 2]
    bins <- seq(min(x), max(x), length.out = my_input() + 1)
    
    # draw the histogram with the specified number of bins
    hist(x, breaks = bins, col = 'darkgray', border = 'white',
         xlab = 'Waiting time to next eruption (in mins)',
         main = 'Histogram of waiting times',
         ylab = paste(my_input(), ' bins'))
    
  })
  
}

```

Note this is a very basic example that doesn't accomplish too much.

To continue the learning process, see these sections from _Mastering Shiny_:

- [Shiny in Action](https://mastering-shiny.org/action-intro.html)
- [Mastering Reactivity](https://mastering-shiny.org/reactivity-intro.html)
- [Best Practices](https://mastering-shiny.org/scaling-intro.html)

# Publishing your Shiny app

In this section we walk through the process of using the RStudio interface to upload a Shiny app to [shinyapps.io](https://shinyapps.io). This requires us to log in and perform a few steps (notably grabbing an authentication token), using the [shinyapps instructions](https://docs.posit.co/shinyapps.io/guide/). Afterwards, we'll be able to publish to _my_account.shinyapps.io/my_project_ (substituting your account name and your project folder name).

Shinyapps.io is free for 5 projects with limited bandwidth. Later we'll see how to completely take control of your own server. The publishing button used in RStudio for shinyapps.io might also be used for in house connection to an organization's own Shiny server (if you have that kind of tech support).

Note that [shinylive](https://posit-dev.github.io/r-shinylive/) is another option and approach to serving up your Shiny content.

# Some extras

Like other widely used ecosystems, Shiny has evolved a variety of tools to extend its functionality and to make working with it easier.

One example is the [_thematic_](https://rstudio.github.io/thematic/) package which will automatically style your plots to match the customised style of your app, using the _thematic_shiny()_ command.

There are other packages like [shinydashboard](https://rstudio.github.io/shinydashboard/) or also [flexdashboard](https://pkgs.rstudio.com/flexdashboard/) which will help you quickly build dashboards.

A [complete list of extensions is here](https://github.com/nanxstats/awesome-shiny-extensions).

# Other topics

This is an introductory workshop, but the _Mastering Shiny_ book and other sites gives many example of ways to improve your coding and make more powerful and complex Shiny apps, via topics like

- [modularizing Shiny code](https://shiny.posit.co/r/articles/improve/modules/)
- [scaling modules](https://mastering-shiny.org/scaling-modules.html)
- [code tips](https://mastering-shiny.org/action-tidy.html) such as the difference between data variables and environmental variables which is important in making reactive functions work properly, for example _diamonds %>% filter(.data[[var]] > .env$min)_
- [debugging](https://mastering-shiny.org/action-workflow.html#action-workflow)
- [reactive graphs](https://mastering-shiny.org/reactive-graph.html) using the _reactlog_ package which shows how the reactive graph evolves over time

# Run your own Shiny server

Shiny gives some information on this at [https://shiny.posit.co/r/deploy](https://shiny.posit.co/r/deploy) and also information on [deploying Python on a Shiny server](https://shiny.posit.co/py/docs/deploy.html). [_Posit Connect_](https://www.rstudio.com/products/connect/) is their commercial offering, but the open source [_Shiny Server_](https://www.rstudio.com/products/shiny/shiny-server/) remains free and open source.

## Digital Ocean

To run one's own server, one has to set up a server someplace.  This is a sketch of how that is done via [DigitalOcean](https://www.digitalocean.com/), which has a straightforward and easy-to-use infrastructure that I would recommend over the complexity and unreliability of AWS.  This will cost at least $4 a month for a minimal setup.

### Get a Droplet

First we set up an account with Digital Ocean, then set up a "Droplet", which in DigitalOcean's lingo is a self-contained server instance.  You should be familiar with the basics of Linux and the command line to manage these steps.

Follow the [instructions to install an Ubuntu or Debian server](https://www.digitalocean.com/community/tutorials/how-to-set-up-an-ubuntu-server-on-a-digitalocean-droplet). Note that other distros are avaialble, but I prefer Debian for its long-term stability and commitment to open source.

### Configure login with SSH

I highly recommend configuring login to the Droplet with SSH keys instead of password access.

These links may be helpful for that:

- [Connect with OpenSSH](https://docs.digitalocean.com/products/droplets/how-to/connect-with-ssh/openssh/)
- [Lost SSH key?](https://docs.digitalocean.com/support/i-lost-the-ssh-key-for-my-droplet/#enable-password-authentication)
- [Troubleshooting SSH](https://docs.digitalocean.com/support/how-to-troubleshoot-ssh-authentication-issues/#fixing-key-permissions-and-ownership)
- [add SSH identity](https://askubuntu.com/questions/20863/how-do-i-permanently-add-an-identity-for-ssh)

### Install Nginx

We need a web server. The [Nginx instructions](https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-20-04
) are straightforward.

Then we should understand how to use the [ufw firewall](https://www.digitalocean.com/community/tutorials/ufw-essentials-common-firewall-rules-and-commands), and then install an [SSL certificate](https://docs.digitalocean.com/support/how-do-i-install-an-ssl-certificate-on-a-droplet/). A modern server should use SSL, since most browsers have begun to reject unencrypted sites by default.

Our Nginx documents (for the home page or other non-Shiny html we want to host) will be put in

_/var/www/html_ on the server, using DigitalOcean's tools, or _scp_ or other transfer/editing methods.

### Install Shiny

Now that the web server is up and running, we can follow the [instructions to install Shiny](https://www.digitalocean.com/community/tutorials/how-to-set-up-shiny-server-on-ubuntu-20-04). There are several steps here that should be followed carefully, in particular to use your own server names where appropriate.

The [complete Shiny server guide is here](https://docs.posit.co/shiny-server/).

### Upload!

The Shiny apps (R files as well as data files) that are served up should be placed in

_/srv/shiny-server/myprojectname_

and your home page should be

_/srv/shiny-server/index.html_

I find it is easiest to just use _git clone_ to pull the files in there. Note that, unlike the _rsconnect_/publish process, there is no generating of a bundle. The Shiny server can work directly with the R files in the directory.

### Final steps

You may also have to configure your nameservers to point to the DigitalOcean droplet if it is just a component of your web services.

My little installation is at [shiny.ryanwomack.com](https://shiny.ryanwomack.com), with a demo project, [CollegeSearchTool](https://shiny.ryanwomack.com/CollegeSearchTool) and actually uses [Plotly](https://plotly.com/) for its graphics.

As always,
*Enjoy R!*
