What is an outbound search
If you're not already familiar with this feature, outbound searches give you the ability to expand your site's search capabilities by allowing users to search for content in various catalogs or services beyond the scope of Stacks' available search integrations and internal search capabilities.
Where search integrations require direct API-based connections (such as EDS, Polaris, Sirsi, etc.), outbound searches are URL-based re-directs that simulate a search in another system without the need of the API. Essentially, outbound searches are a set of rules that tell your Stacks site how to recognize searches going out to a database, catalog, other website or service outside of Stacks.
How an outbound search works
Outbound searches use different components of a URL to identify where the search should go out to, how to perform the search, the search term(s), and can also include other information if logging in to that service is required for a user to see the results.
After an outbound search has been configured and enabled to appear with your primary search bar, users can select the search type and enter their search term. Once the search is initiated a new browser tab will open, taking the user to the catalog or service your search is configured for. The search term will be carried over and the results displayed in that catalog or service.
How Stacks thinks about URLs
URLs are daunting because they can be very long strings of words and characters that can appear as (and are) a different language to most of us. Rather than thinking of them as such, rest assured a URL is really just a set of building blocks that tell your browser to do something.
While there will be some parts of a URL that change every time, they will also have predictable patterns. It's these patterns we will use to configure the search, while ignoring the parts that will change.
Anatomy of a URL
Taking a closer look at a sample URL, let's break it down into some basic components, many of which we would use to setup an outbound search; we'll use this as our example:
https://www.ebscohost.com/ehost/results?q=climate+change
Part |
Where in the URL is this? |
Why does this matter? |
Protocol |
https:// |
This actually doesn't matter, it's almost always ignored |
Domain |
www.ebscohost.com |
This is usually the anchor, it tells where the search is going out to |
Path |
/ehost/results |
This is often the most important, it tells how to perform the search |
Parameters |
?q=climate+change |
This is usually trimmed or generalized, "climate+change" for example are the search terms, this will change with every search |
What to keep, what to ignore?
While each URL can be different, generally speaking here are some basic rules to follow.
What you would normally keep:
- Domain
- Stable path elements (those that don't change with every search)
- Database or other platform identifiers
What you almost always ignore:
- Session IDs (this will expire or change)
- Search terms
- User or institution identifiers
- Delimiters
- Anything that changes with every new search
Deconstructing a URL
To setup an outbound search you will need to run a couple of successful searches in the service you are using. Then compare the URLs to see what changes and what doesn't, so you can deconstruct the search URL into the components we identified above.
Let's look at an example of breaking down a URL, we'll use the WorldCat online catalog search for this.
If I navigate to https://search.worldcat.org/ and search 'dogs' I can see this returns a successful search with the following URL in my browser bar:
https://search.worldcat.org/search?q=dogs&offset=1
If we search again, this time 'cats,' we can see the URL is virtually identical:
https://search.worldcat.org/search?q=cats&offset=1
Comparing the two URLs allows us to see some of the components we were talking about earlier as far as what we will most likely need to keep to setup an outbound search:
https://search.worldcat.org/search?q=dogs&offset=1
https://search.worldcat.org/search?q=cats&offset=1
Part |
Where is this in the URL |
Protocol |
https:// |
Domain |
search.worldcat.org |
Path |
/search |
Parameter |
?q=cats&offset=1 |
As we saw in the first example, the '?' in the parameter is a delimiter, separating the the path from the query string, along with the '=' being the delimiter between the query string and the search value.
This time we also see '&' which is another delimiter, this time between multiple parameters in the query string, 'q' and 'offset=1.' Where 'offset=1' is used to tell the server to skip the first result and start at the second item, this is often used as a form of pagination for large datasets.
Other common characters you may see can include:
- '+' or '%20' for spaces
- other URL-encoded characters for puncuation
Stacks will handle most encoding automatically, but it's still good to know that this matters.
Understanding all of these URL components is key, because stable paths and parameters are the backbone of a reliable outbound search. From time to time vendors may redesign their URLs, this would break your outbound search; so regular testing and knowing how the URL works will be key to fixing it should it happen to you.
Putting it all together
Let's see how this all goes together when it comes to configuring an outbound search in Stacks. Here, we are going to discuss what fields you will see for this configuration, along with inputing the example WorldCat URL we saw above to see how this all goes.
For more detailed step-by-step instructions, please see the Outbound Search user guide article.
There are 5 fields you will see when adding new Outbound Searches in Stacks
- Label
- The name of the search, this is what your users/patrons will see when selecting which search they want to use.
- Example: WorldCat
- The name of the search, this is what your users/patrons will see when selecting which search they want to use.
- URL
- This is where you would enter the protocol, domain, and path for the search catalog/service
- Example: https://search.worldcat.org/search
- This is where you would enter the protocol, domain, and path for the search catalog/service
- Search Parameter
- This field identifies the search parameter
- Example: q
- This field identifies the search parameter
- Extra Parameters
- Extra parameters are things like the offset we saw in our example, but could also include information that will prompt the user to login to access the catalog or service
- Example: offset=1
- Extra parameters are things like the offset we saw in our example, but could also include information that will prompt the user to login to access the catalog or service
- Fragment
- Fragments will appear at the end of a URL and start after the '#' icon, this usually acts as a bookmark for the browser to go to a specific place on the page. While you may see and need this, it is not used very often
Common errors and additional tips
Some common errors that we typically find include:
- Using a URL that includes a session ID - the search may work initially, but will break once the time expires as identified by the session ID
- Copying a URL from a 'logged-in' view - this can also produce an error due if the authentication parameters are not added
- Using a POST-based search - behind the scenes, searches typically work by using a GET function to request data, whereas POST will typically send data. While POST could be used for searching, its not as common and cannot be replicated with an outbound search
- Using a vendor that requires JavaScript-based search forms
Additional tips to consider:
- Important! Sometimes it takes a few tries and some trial and error to setup an outbound search. If your first attempt doesn't work, re-evaluate how you broke down the URL; you may need to try a few variations of values in the URL, search parameter, and extra parameters fields
- As mentioned earlier, search vendors may re-design their platform or service resulting in changes to the URL. Regular testing of outbound searches can be beneficial so you can update configuration if the URL changes
- If the search stops working or produces an error, and especially if you have extra parameters, double check those values to make sure there is no session ID information or an issue with the login prompts