Mastering Outbound Searches – Stacks for Libraries

What is an outbound search

If you're not already familiar with this feature, outbound searches give you the ability to expand your site's search capabilities by allowing users to search for content in various catalogs or services beyond the scope of Stacks' available search integrations and internal search capabilities.

Where search integrations require direct API-based connections (such as EDS, Polaris, Sirsi, etc.), outbound searches are URL-based re-directs that simulate a search in another system without the need of the API. Essentially, outbound searches are a set of rules that tell your Stacks site how to recognize searches going out to a database, catalog, other website or service outside of Stacks.

This opens your options to ensure your users can still access content from free or paid subscriptions and other sources you have access to.

How an outbound search works

Outbound searches use different components of a URL to identify where the search should go out to, how to perform the search, the search term(s), and can also include other information if logging in to that service is required for a user to see the results.

After an outbound search has been configured and enabled to appear with your primary search bar, users can select the search type and enter their search term. Once the search is initiated a new browser tab will open, taking the user to the catalog or service your search is configured for. The search term will be carried over and the results displayed in that catalog or service.

How Stacks thinks about URLs

URLs are daunting because they can be very long strings of words and characters that can appear as (and are) a different language to most of us. Rather than thinking of them as such, rest assured a URL is really just a set of building blocks that tell your browser to do something.

While there will be some parts of a URL that change every time, they will also have predictable patterns. It's these patterns we will use to configure the search, while ignoring the parts that will change.

Anatomy of a URL

Taking a closer look at a sample URL, let's break it down into some basic components, many of which we would use to setup an outbound search; we'll use this as our example:

https://www.ebscohost.com/ehost/results?q=climate+change

Part	Where in the URL is this?	Why does this matter?
Protocol	https://	This actually doesn't matter, it's almost always ignored
Domain	www.ebscohost.com	This is usually the anchor, it tells where the search is going out to
Path	/ehost/results	This is often the most important, it tells how to perform the search
Parameters	?q=climate+change	This is usually trimmed or generalized, "climate+change" for example are the search terms, this will change with every search

It's important to note, the '?' in the parameter is a delimiter. In the case of our search, it separates the the 'path' from the 'query string,' which is the parameter that tells how to search. Similarly, the '=' is the delimiter that separates the query string and the search value.

What to keep, what to ignore?

While each URL can be different, generally speaking here are some basic rules to follow.

What you would normally keep:

Domain
Stable path elements (those that don't change with every search)
Database or other platform identifiers

What you almost always ignore:

Session IDs (this will expire or change)
Search terms
User or institution identifiers
Delimiters
Anything that changes with every new search

Deconstructing a URL

To setup an outbound search you will need to run a couple of successful searches in the service you are using. Then compare the URLs to see what changes and what doesn't, so you can deconstruct the search URL into the components we identified above.

Let's look at an example of breaking down a URL, we'll use the WorldCat online catalog search for this.

If I navigate to https://search.worldcat.org/ and search 'dogs' I can see this returns a successful search with the following URL in my browser bar:

https://search.worldcat.org/search?q=dogs&offset=1

If we search again, this time 'cats,' we can see the URL is virtually identical:

https://search.worldcat.org/search?q=cats&offset=1

Comparing the two URLs allows us to see some of the components we were talking about earlier as far as what we will most likely need to keep to setup an outbound search:

https://search.worldcat.org/search?q=dogs&offset=1

https://search.worldcat.org/search?q=cats&offset=1

Part	Where is this in the URL
Protocol	https://
Domain	search.worldcat.org
Path	/search
Parameter	?q=cats&offset=1

As we saw in the first example, the '?' in the parameter is a delimiter, separating the the path from the query string, along with the '=' being the delimiter between the query string and the search value.

This time we also see '&' which is another delimiter, this time between multiple parameters in the query string, 'q' and 'offset=1.' Where 'offset=1' is used to tell the server to skip the first result and start at the second item, this is often used as a form of pagination for large datasets.

Other common characters you may see can include:

'+' or '%20' for spaces
other URL-encoded characters for puncuation

Stacks will handle most encoding automatically, but it's still good to know that this matters.

Understanding all of these URL components is key, because stable paths and parameters are the backbone of a reliable outbound search. From time to time vendors may redesign their URLs, this would break your outbound search; so regular testing and knowing how the URL works will be key to fixing it should it happen to you.

Putting it all together

Let's see how this all goes together when it comes to configuring an outbound search in Stacks. Here, we are going to discuss what fields you will see for this configuration, along with inputing the example WorldCat URL we saw above to see how this all goes.

For more detailed step-by-step instructions, please see the Outbound Search user guide article.

There are 5 fields you will see when adding new Outbound Searches in Stacks

Label
- The name of the search, this is what your users/patrons will see when selecting which search they want to use.
  - Example: WorldCat
URL
- This is where you would enter the protocol, domain, and path for the search catalog/service
  - Example: https://search.worldcat.org/search
Search Parameter
- This field identifies the search parameter
  - Example: q
Extra Parameters
- Extra parameters are things like the offset we saw in our example, but could also include information that will prompt the user to login to access the catalog or service
  - Example: offset=1
Fragment
- Fragments will appear at the end of a URL and start after the '#' icon, this usually acts as a bookmark for the browser to go to a specific place on the page. While you may see and need this, it is not used very often

Common errors and additional tips

Some common errors that we typically find include:

Using a URL that includes a session ID - the search may work initially, but will break once the time expires as identified by the session ID
Copying a URL from a 'logged-in' view - this can also produce an error due if the authentication parameters are not added
Using a POST-based search - behind the scenes, searches typically work by using a GET function to request data, whereas POST will typically send data. While POST could be used for searching, its not as common and cannot be replicated with an outbound search
Using a vendor that requires JavaScript-based search forms

Additional tips to consider:

Important! Sometimes it takes a few tries and some trial and error to setup an outbound search. If your first attempt doesn't work, re-evaluate how you broke down the URL; you may need to try a few variations of values in the URL, search parameter, and extra parameters fields
As mentioned earlier, search vendors may re-design their platform or service resulting in changes to the URL. Regular testing of outbound searches can be beneficial so you can update configuration if the URL changes
If the search stops working or produces an error, and especially if you have extra parameters, double check those values to make sure there is no session ID information or an issue with the login prompts