What is slack

Slack is a cloud-based set of team collaboration tools and services, founded by Stewart Butterfield. Slack began as an internal tool used by their company, Tiny Speck, in the development of Glitch, a now defunct online game. The name is an acronym for "Searchable Log of All Conversation and Knowledge".¹

What is webhook

A webhook in web development is a method of augmenting or altering the behavior of a web page, or web application, with custom callbacks. These callbacks may be maintained, modified, and managed by third-party users and developers who may not necessarily be affiliated with the originating website or application. The term "webhook" was coined by Jeff Lindsay in 2007 from the computer programming term hook.²

Slack's webhook

Slack offers two kinds of webhooks to allow the communication with the slack channels.

1. Incoming Webhooks

Incoming Webhooks are a simple way to post messages from external sources into Slack. They make use of normal HTTP requests with a JSON payload that includes the message text and some options. Message Attachments can also be used in Incoming Webhooks to display richly-formatted messages that stand out from regular chat messages.

2. Outgoing webhooks

Outgoing webhooks will send an HTTP POST request to your specified URL when a message matches one or both of the following conditions:

• The message is in the specified channel
• The message begins with one of the defined trigger word(s)

This is useful for providing automated responses to messages your team members post. For example, you might have a service that posts the status of a server. This service could receive the HTTP POST from Slack and automatically respond with a status update.

The outgoing webhook integration is only available in public channels. If you would like to get data out of private groups and DMs in real-time, try a slash command.

To summarize, incoming webhooks allows you to send messages to a slack channel; while outgoing webhooks allows you to get some message from a slack channel and send responses to it.

How a slack robot works using webhooks

A slack robot is basically a piece of software that

• takes some informaton from either external source(e.g. a web crawler) of from a slack channel(using the outgoing webhook),
• process the information and send some information back to the slack channel(using either the incoming webhook or the response function of the outgoing webhook).

Now let's build a simple slack bot that looks at stackoverflow's question page every minute, grab the current number of total questions, and return this information to our slack channel using incoming webhook.

1. Add incoming webhooks integration in slack

Go to slacks' incoming webhook integration page to choose a slack channel and click the Add Incoming WebHooks Integration button to add a new incoming webhook of your own.

Copy the generated WebHook URL and store it somewhere, you will need it later. You can also change the name/icon/description of your slack bot here.

2. Create a simple web crawler in python

Now we will create a python program to do the web crawling and send the message. For html parsing, we will be using BeautifulSoup4 and lxml here. Install it using the following command if you don't already them installed:

pip install beautifulsoup4 lxml

Now create a new python script file, scraper.py like this:
First import all needed libraries.

# Import all needed libraries
from bs4 import BeautifulSoup as bs
import urllib.request
import threading
import requests
import json
import time

Then add a function called checkStatus to do web crawling.

def checkStatus():
    # Function that checks a specific webpage and returns the information as string.
    url = 'https://stackoverflow.com/questions' # The URL of sltackoverflow.
    headers = {
                "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0",
                } # Add browser header to your http request to avoid blocking.
    req = urllib.request.Request(url=url, headers=headers)
    try:
        page = urllib.request.urlopen(req)
    except:
        return '503: service unavailable, try again later.'
    soup = bs(page, "lxml")
    count = soup.find("div", class_='summarycount al')
    return count.string

Then add a function called notify to send data to your slack channel using incoming webhook.

def notify(data):
    # Function that send data in a string to your slack channel using incoming webhook
    slack_url = '[PASTE YOUR GENERATED INCOMING WEBHOOK URL HERE]'
    requests.post(slack_url, data = json.dumps({
            'text': data,
            'username': 'Tornado-webhook-notify',
            'icon_emoji': ':bulb:',
            'link_names': 1,
        }))

Create a python thread class to be able to run some function at some specified time interval.

# Create a recursive thread to run the checkup every 1 minute.
class RecursiveThread(threading.Thread):
    def __init__(self, event):
        threading.Thread.__init__(self)
        self.stopped = event

    def run(self):
        # Set the number to 60.0 to run every 1 minute.
        while not self.stopped.wait(60.0): 
            status = checkStatus()
            notify('The total question number is: ' + status)

Then at last call these functions and classes to check the web site for 3 minutes.

if __name__ == '__main__':
        # Create the event used to stop the thread.
        stopFlag = threading.Event() 
        # Instantialize the thread.
        thread = RecursiveThread(stopFlag)
        # Start the thread.
        thread.start()
        # Wait for some time(3 minutes).
        time.sleep(180.0)
        # Use the flag to stop the thread.
        stopFlag.set()

If successful, you shoud be able to see the message telling you the current stackoverflow question number 3 times in your slack channel.

Add more control using outgoing webhooks

The above slack bot is already quite practically useful(you can do a bunch of things like auto checking the price of a apecific item in an online store and such folloing the same logic), but to add more control of the bot behavior, we need the outgoing webhooks. In order to use outgoing webhook, you need a server machine that can be accessed from the internet. Once you have the server setup, you only need to add a server logic that listens to the POST request from your slack channel and responses to it. Python frameworks like Tornado and Django can help you do this relatively easily.
Setting up a server to handle RESTful APIs is a rather broad topic and I won't do it here. It also seems that slack is deprecating the outgoing webhooks as well.

Maybe I will write about building slack apps using their latest APIs in the future. Also I think it is noteworthy that there are alternatives to slack like Rocket.Chat that allows the user to store their messages and information in their own servers for credential reasons(this is impossible with slack). So maybe it's better to these alternatives instead.