```{r setup, include=FALSE}
knitr::opts_chunk$set(
  results = 'asis',
  echo = FALSE,
  warning = FALSE,
  message = FALSE,
  fig.align = 'center'
)

#Load libraries
library(tidyverse)
library(gapminder)
library(kableExtra)
library(readxl)
library(cowplot)
library(lubridate)

#setup tidy data

#Read data from data/comissioning_data.xlsx. This file contains four sheets each of which contain the data from a 
#single sample type. SR0204 is a high reactivty control sample, SR0190 is a low reactivity control sample SR0170 is a medium reactivity control sample and 
#Other are test samples that have been included in project but may not be used for assessing the effect of process
#automation on test results.The results for each sample include test results as well as process data. 

SR0204_data<- read_excel("data/comissioning_data.xlsx",
                         sheet="SR0204",
                         col_names=TRUE,
                         col_type=c("text","text","text","date","numeric","numeric","numeric","numeric","numeric","numeric","numeric","numeric","numeric","date","numeric","date","numeric","date","numeric","date","date","date","text")) #reads data in sheet SR0204 of the file comissioning_data.xlsx, column types have been specified to facilitate binding rows.

SR0190_data<- read_excel("data/comissioning_data.xlsx",
                         sheet="SR0190",
                         col_names=TRUE,
                         col_type=c("text","text","text","date","numeric","numeric","numeric","numeric","numeric","numeric","numeric","numeric","numeric","date","numeric","date","numeric","date","numeric","date","date","date","text")) #reads data in sheet SR0190 of the file comissioning_data.xlsx, column types have been specified to facilitate binding rows.

SR0170_data<- read_excel("data/comissioning_data.xlsx",
                         sheet="SR0170",
                         col_names=TRUE,
                         col_type=c("text","text","text","date","numeric","numeric","numeric","numeric","numeric","numeric","numeric","numeric","numeric","date","numeric","date","numeric","date","numeric","date","date","date","text")) #reads data in sheet SR0170 of the file comissioning_data.xlsx, column types have been specified to facilitate binding rows.

#create a single data frame and add the following columns of test results test_number,final_mass,mass_loss,CRI and CSR. 

all_data<- bind_rows(SR0204_data,SR0190_data,SR0170_data)%>% # binds three data frames SR0204_data, SR0190_data and SR0170 into a single data frame and assigns the variable all_data
  mutate(test_number= row_number(), #Add a column that gives each line a unique identifier
         final_mass= `Weight+10mm` + `Weight-10mm`, # calculates a new column called final_mass by adding `Weight+10mm` and `Weight-10mm`columns.
         mass_loss=Weight_Out - final_mass, # calculates a new column called mass_loss by subracting Weight_out from final_mass
         CRI= (((Weight_In-Weight_Out)/Weight_In)*100), #calculates a new column called CRI using formula ((Weight_In - Weight_Out)/Weight_In) *100
         CSR= ((`Weight+10mm`/Weight_Out)*100), #calculates a new column called CSR by using formula (`Weight+10mm`/Weight_Out)*100
         PID_response= (time_return_range - Runtime_switch_CO2))%>% # calculates a new column called PID_Response by subracting the time that the sample temperatured returned to the range and the time that the sample atmosphere swtiched to CO2
   view()#views the dataframe

#Import data for example process plot 
raw_process<-read_tsv("data/example_raw_test_data.prn",col_names=TRUE,skip=4)%>% #read data from file data/example_raw_test_data.prn, skipping the first 4 rows as they contain metadata, assigns dataframe to raw_process
  mutate(elapsed = seconds(row_number()-1)) #adds column elapsed by numbering the rows, because we know that an observation has been taken every second for the duration of the test. 
```

# Introduction
I am a Senior Research Scientist in the Coke Making Team at QCAT in QLD. My team operates two pilot scale coke ovens and characterises the coke that we produce to support metallurgical coal producers with their marketing efforts. My team deals with quite a large amount of data in excel to facilitate the reporting of process data and coke test results to our clients. Prior to data school I had no experience in coding in R. 

# My Project
<div class= "col-md-8"> 

The Coke Reactivity Test is routinely carried out at QCAT by the coke making team to assess the quality of coke produced in the Research Coke Oven and Non Recovery Coke Oven. 

A 20mm sample of coke is reacted with Carbon Dioxide for a period of two hours at a temperature of 1100oC, The mass loss is indicated by the Coke Reactivity Index (CRI). The sample is then tumbled to determine the Coke Strength after Reaction (CSR) which is the mass of coke that remains greater than 10mm in size. The indices indicate the behavior of coke in the iron-making blast furnace. A coke that has a low CRI and high CSR is highly regarded in the market.

My project is to validate the implementation of automated temperature control for the Coke Reactivity Test. ***The questions that we set out to answer through validation testing were; Is there an improvement in the repeatability of the test results? and Is there a difference in the test results between manual and automated control?***

The data collected for this project was both test result data recorded in excel and process data extracted from the control system as a text file. Figure \@ref(fig:Process-data-plot) shows the temperature of the sample over the length of the test, the temperature must be controlled within a tight range which is why a PID control system is useful.
</div>

<div class= "col-md-4">  
![](resources/img/DanCSR.jpg){width=220px, out.extra='frontpage'}
</div>

```{r Process-data-plot, out.width='60%', fig.align='center', fig.height= 4, fig.width=6, fig.cap="Example process plot"}
ggplot (data=raw_process, 
        mapping=aes(x= elapsed,
        y= Sample_Temp)) +
  geom_point(colour= "dark blue", size = 1)+
  geom_hline(aes(yintercept=1097),colour= "red")+
  geom_hline(aes(yintercept=1103),colour= "red")+
  ylim(600,1200)+
  theme_linedraw() +
  theme(legend.position = "bottom")+
  labs(x="Time Elapsed (Seconds)", y="Sample Temperature oC",caption = "Temperature control limits indicated by red lines")

```

## Preliminary results

Three coke samples with different reactivity were included in the validation program. Figure \@ref(fig:CRI-vs-CSR-plot) (below) shows the distinct groupings of results, at first glance it also shows that the grouping of results for tests run in automated mode are tighter, indicating that the automated method improves the repeatability of the results. A student's t- test should be used to determine if the differences in the means of the CRI and CSR between the automated and manual methods for each sample are significant. 

```{r CRI-vs-CSR-plot, out.width='60%', fig.align='center', fig.height= 4, fig.width=6, fig.cap="CRI vS CSR for Control Samples"}
ggplot (data=all_data, 
        mapping=aes(x= CRI,
                    y= CSR,
                    colour=Mode)) +
  geom_point(aes(shape=Sample),size=2) +
  geom_abline(aes(intercept=100.34,slope=-1.34), colour = "dark green")+
  scale_color_brewer(palette = "Set1") +
  theme_linedraw() +
  theme(legend.position = "bottom") +
  labs(caption= "The green line represents the relationship CSR= 100.34 -1.34*CRI (Koval, Sakurovs & Hockings 2018)") 
```
Further plotting in Figure \@ref(fig:CRI-vs-CSR-plot-operator) (below) showed that there is an operator effect, this is not an unexpected finding as in the manual method the temperature is controlled by the operator. An additional source of variation in test results between a single coke is sample selection. 

```{r CRI-vs-CSR-plot-operator, out.width='60%', fig.align='center', fig.height= 4, fig.width=6, fig.cap="CRI vS CSR for Control Samples by operator"}
ggplot (data=all_data, 
        mapping=aes(x= CRI,
        y= CSR,
        colour=Operator)) +
  geom_point(aes(shape=Mode),size=2) + 
  scale_color_brewer(palette = "Set2") +
  theme_linedraw() +
  theme(legend.position = "bottom")
```
# My Digital Toolbox

Throughout the course of Data School I have learned to use the Tidyverse and the packages within it (e.g.dplyr, readr and tidyr) as well as readxl to import and manipulate data. I have also used ggplot2 to visualise my data. Additionally I have learnt how to use of Git and Github to track changes and collaborate on R projects. I also discovered a fun function called `beep()` that makes a noise when a script has sucessfully executed. 

# My time went ...

I spent quite a bit of time tidying and joining my data into a single data frame using the `bind_rows()` functions. I also added additional columns of calculated results data using the `mutate()` function. I found it most challenging to deal with time series data, using the Lubridate package. 

# Next steps

I'd like to develop my skills in R coding further and utilise R to conduct data analysis and to generate plots in future research projects (and to finalise the commissioning of the automated control system). I'm also interested in using R to make it easier and quicker to plot and present untidy data collected by my team to monitor equipment over time. 

# My Data School Experience

I have enjoyed the experience of learning in a supportive and collaborative environment with people so willing to share their skills, the other really wonderful aspect of data school is that it brings together a diverse group of people from across the country and organisation. I have also  appreciated the opportunity to spend dedicated time on learning a skill that will set me up for success as a researcher and team leader. 

