aws s3のアクセスログをslackへ飛ばす

s3のイベントでは、boto3を使用したオブジェクトの書き込みが拾えないため、アクセスログをslackへ飛ばす方法を試行錯誤してみました。

s3のアクセスログをs3に格納して、オブジェクトの作成イベントでlambdaを起動して、ログ内容を解析しslackへ飛ばす手順です。

まずは、s3のアクセスログ保管用のバケットを作成。

ターゲットのs3バケットのプロパティで「サーバーアクセスのログの記録」で出力を設定。

lambdaを作成

ひな形は、設計図 cloudwatch-alarm-to-slack-python を使用して作成してください。

下のコードに置き換える。

kmsの設定は、ほかのブログを参考にしてください。

トリガーにs3を追加して、s3のアクセスログ保管用のバケットを指定して、obejctcreatedを選択してください。

'''
Follow these steps to configure the webhook in Slack:

  1. Navigate to https://&lt;your-team-domain&gt;.slack.com/services/new

  2. Search for and select "Incoming WebHooks".

  3. Choose the default channel where messages will be sent and click "Add Incoming WebHooks Integration".

  4. Copy the webhook URL from the setup instructions and use it in the next section.

To encrypt your secrets use the following steps:

  1. Create or use an existing KMS Key - http://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

  2. Click the "Enable Encryption Helpers" checkbox

  3. Paste &lt;SLACK_CHANNEL&gt; into the slackChannel environment variable

  Note: The Slack channel does not contain private info, so do NOT click encrypt

  4. Paste &lt;SLACK_HOOK_URL&gt; into the kmsEncryptedHookUrl environment variable and click encrypt

  Note: You must exclude the protocol from the URL (e.g. "hooks.slack.com/services/abc123").

  5. Give your function's role permission for the kms:Decrypt action.

     Example:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1443036478000",
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt"
            ],
            "Resource": [
                "&lt;your KMS key ARN&gt;"
            ]
        }
    ]
}
'''

import boto3
import json
import logging
import os
import datetime
import calendar

from base64 import b64decode
from urllib.request import Request, urlopen
from urllib.error import URLError, HTTPError


# The base-64 encoded, encrypted key (CiphertextBlob) stored in the kmsEncryptedHookUrl environment variable
ENCRYPTED_HOOK_URL = os.environ['kmsEncryptedHookUrl']
# The Slack channel to send a message to stored in the slackChannel environment variable
SLACK_CHANNEL = os.environ['slackChannel']

HOOK_URL = "https://" + boto3.client('kms').decrypt(CiphertextBlob=b64decode(ENCRYPTED_HOOK_URL))['Plaintext'].decode('utf-8')

logger = logging.getLogger()
logger.setLevel(logging.INFO)


def lambda_handler(event, context):
    logger.info("Event: " + str(event))

    s_flag = 0    
    if 'eventSource' in event['Records'][0]:
        if event['Records'][0]['eventSource'] == 'aws:s3':
            reason = event['Records'][0]['s3']
            bucket = event['Records'][0]['s3']['bucket']['name']
            key = event['Records'][0]['s3']['object']['key']
            client = boto3.client('s3')
            
            try:
                response = client.get_object(Bucket=bucket,Key=key)
            except Exception as e:
                logger.error("Failed to get object: {}".format(e))
            s3txt=response['Body'].read().decode('utf-8')
            logger.info(s3txt)
            s3l = s3txt.splitlines()
            for s3dt in s3l:
                s3items = s3dt.split(" ")
                logger.info(s3items)
                tbucket = s3items[1]
                ttype = s3items[7]
                logger.info(ttype)
                if ttype == 'REST.PUT.OBJECT':
                    tkey = s3items[8]
                    tresult = s3items[12]
                    ttime = s3items[2] + s3items[3]
                    slack_message = {
                        'channel': SLACK_CHANNEL,
                        'username': 'S3bucket',
                        'pretext': "",
                        'color': "good",
                        'text': "%s %s %s %s" % (ttime, tresult, tbucket, tkey)
                    }
                    s_flag = 1    
                    
                
        
    if s_flag == 1:
        req = Request(HOOK_URL, json.dumps(slack_message).encode('utf-8'))
        try:
            response = urlopen(req)
            response.read()
            logger.info("Message posted to %s", slack_message['channel'])
        except HTTPError as e:
            logger.error("Request failed: %d %s", e.code, e.reason)
        except URLError as e:
            logger.error("Server connection failed: %s", e.reason)

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

'''

Follow these steps to configure the webhook in Slack:

1. Navigate to https://<your-team-domain>.slack.com/services/new

2. Search for and select "Incoming WebHooks".

3. Choose the default channel where messages will be sent and click "Add Incoming WebHooks Integration".

4. Copy the webhook URL from the setup instructions and use it in the next section.

To encrypt your secrets use the following steps:

1. Create or use an existing KMS Key - http://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

2. Click the "Enable Encryption Helpers" checkbox

3. Paste <SLACK_CHANNEL> into the slackChannel environment variable

Note: The Slack channel does not contain private info, so do NOT click encrypt

4. Paste <SLACK_HOOK_URL> into the kmsEncryptedHookUrl environment variable and click encrypt

Note: You must exclude the protocol from the URL (e.g. "hooks.slack.com/services/abc123").

5. Give your function's role permission for the kms:Decrypt action.

Example:

{

"Version": "2012-10-17",

"Statement": [

{

"Sid": "Stmt1443036478000",

"Effect": "Allow",

"Action": [

"kms:Decrypt"

"Resource": [

"<your KMS key ARN>"

]

}

]

}

'''

import boto3

import json

import logging

import os

import datetime

import calendar

from base64 import b64decode

from urllib.request import Request, urlopen

from urllib.error import URLError, HTTPError

# The base-64 encoded, encrypted key (CiphertextBlob) stored in the kmsEncryptedHookUrl environment variable

ENCRYPTED_HOOK_URL = os.environ['kmsEncryptedHookUrl']

# The Slack channel to send a message to stored in the slackChannel environment variable

SLACK_CHANNEL = os.environ['slackChannel']

HOOK_URL = "https://" + boto3.client('kms').decrypt(CiphertextBlob=b64decode(ENCRYPTED_HOOK_URL))['Plaintext'].decode('utf-8')

logger = logging.getLogger()

logger.setLevel(logging.INFO)

def lambda_handler(event, context):

logger.info("Event: " + str(event))

s_flag = 0

if 'eventSource' in event['Records'][0]:

if event['Records'][0]['eventSource'] == 'aws:s3':

reason = event['Records'][0]['s3']

bucket = event['Records'][0]['s3']['bucket']['name']

key = event['Records'][0]['s3']['object']['key']

client = boto3.client('s3')

try:

response = client.get_object(Bucket=bucket,Key=key)

except Exception as e:

logger.error("Failed to get object: {}".format(e))

s3txt=response['Body'].read().decode('utf-8')

logger.info(s3txt)

s3l = s3txt.splitlines()

for s3dt in s3l:

s3items = s3dt.split(" ")

logger.info(s3items)

tbucket = s3items[1]

ttype = s3items[7]

logger.info(ttype)

if ttype == 'REST.PUT.OBJECT':

tkey = s3items[8]

tresult = s3items[12]

ttime = s3items[2] + s3items[3]

slack_message = {

'channel': SLACK_CHANNEL,

'username': 'S3bucket',

'pretext': "",

'color': "good",

'text': "%s %s %s %s" % (ttime, tresult, tbucket, tkey)

}

s_flag = 1

if s_flag == 1:

req = Request(HOOK_URL, json.dumps(slack_message).encode('utf-8'))

try:

response = urlopen(req)

response.read()

logger.info("Message posted to %s", slack_message['channel'])

except HTTPError as e:

logger.error("Request failed: %d %s", e.code, e.reason)

except URLError as e:

logger.error("Server connection failed: %s", e.reason)