What is a Web Data Connector (WDC)?
Tableau defines a WDC in the following way:
To see the WDC documentation see here: https://tableau.github.io/webdataconnector/docs/
Why would you want to use a WDC?
For me, I wanted to access Xero data from Tableau. To automate this process, we would need to use a WDC to connect to the Xero API and request the data.
Here are 2 main reasons why a WDC might be needed:
- To connect to data that is not easily accessible i.e. an API
- We can connect to API’s using tools such as Alteryx – an analytics tools. However, not everyone has access to Alteryx and even if they did this is not always possible because of the authentication process that some API’s might require you to go through. For example, it might require an authentication token to be processed before we can get access to the data.
What are cookies?
With quarantine I have made about a lifetime’s worth of these types of cookies. But unfortunately, this blog talks about a different kind of cookie: HTTP cookies.
HTTP cookies store data about a user’s activity and interaction with a website. It can be used by companies to track customer activity. It can hold information such as recently visited sites or even login information.
This is useful for the authentication process as we can send login information to our server to then request access to data via an API.
Let’s dive in!
In this example, we are going to look at the Xero API, which allows us access to financial data. See their documentation here: https://developer.xero.com/documentation/
When authenticating, an important concept to understand is the difference between server side and client side in web applications. Client side code controls what happens on the web page that the user is interacting with. Server side code processes what is happening in the web server and is therefore not accessible by the user. This is important when dealing with sensitive data such as login details or access token as we don’t want to process this on the client side.
This is an important concept to understand because when we are authenticating a user to get access to an API, we need to use both client and server side code.
The logic is the following:
- The client side will allow us to get credentials from a user which we will send to the API via a POST request.
- Xero will confirm that the credentials are correct, the user exists and that the user allows permission for a third party like our WDC to get access to the data.
- We will receive a token that we can then use to make a request to get access to data. We need to process this token and the data in the server side.
In the server side code, we using a package called Express JS to create the routes below.
In our server side code we’ve got to create routes to process different information from different pages.
We need the following routes:
The first route leads us to a html page that you first need to create. It acts as the front end of our web application. We want our users to input their information in there.
Our very basic html page looks like this:
It contains a form for users to input their credentials and a drop down for the user to select which type of data they want to get from the API.
We then need to create a route in the server side to send you to the html file that you specify:
Here we are saying that we want to look for the connect button. Upon clicking that button, we want to do various things, i.e store the data in an object, console.log it and then call on another function.
This other function will re-direct us to the login page. That function is outlined below:
The first part of the function defines what we want to send to the server. In this case it is the credentials.
The second part uses AJAX to actually send the credentials to the server.
Notice that the URL I am sending this data to has an endpoint of “/login”. This is important because this is the route in the server side that we want to send our credentials to and then process that.
Now let’s switch to our server side to see what is happening with those credentials.
Notice that after each “app.” we specify a request method. Since we are now in server side, we are dealing with routes, and each route does something different. For example the “/login” route is a post request that receives the credentials that has been “posted” to the server side from the client side.
It then goes through various other routes such as “/connect” and “/callback”. These will change depending on your API, so it’s important to read the documentation to get these right.
There are a few things in this code that I want to bring your attention to.
- Getting the user to login to Xero so that Xero can confirm that they have access to the data.
This bit of code will redirect the user to the login page of Xero and will ask the user to specify which data they want access to. It will look something like this after the user logs in.
Once the user allows access, it will provide us a token in the “/callback” route.
The “/callback” route is where we will receive the access token and process it using cookies.
Here we are using an if-statement to say to check whether or not we have an access token and defines what we need to do with either outcomes.
If we have an access token we want to store this in a cookie.
We can do that by using
We then want to console.log it to check that we actually do have the access token stored in the cookie.
If we don’t have an access token, we want to redirect the user back to the html page where they can input their credentials and go through the process of logging into Xero and giving their permission to get the data.
Back to client side
Since we have our access token stored in the cookie, we can now refer to that in the client side code.
We can then use that access token in our WDC to get access to our data.
To get access to our data, we need to construct a URL with the access code stored as a parameter. Again, this is part of the documentation with the Xero API, so it might be different for other API’s.
In the Web Data Connector, we need to call on the URL in the “getData” part.
Here you can see that we constructed the URL and stored it in a variable. We then use that URL to get the data that we want.
This is where cookies can really come in handy. It allows us to send and receive data from the client to the server. In this case, that is the access token!
Note: I am working on a second version, where the user will not need to add their “consumerKey” or “consumerSecret”.