discord-downloader-go

Discord Downloader 💾

<a href="https://github.com/get-got/discord-downloader-go/releases" alt="All Releases">
    <img src="https://img.shields.io/github/downloads/get-got/discord-downloader-go/total?label=all-releases&logo=GitHub" />
</a>
<a href="https://github.com/get-got/discord-downloader-go/releases/latest" alt="Latest Release">
    <img src="https://img.shields.io/github/downloads/get-got/discord-downloader-go/latest/total?label=latest-release&logo=GitHub" />
</a>
<br>
<a href="https://travis-ci.com/get-got/discord-downloader-go" alt="Build Status">
    <img src="https://travis-ci.com/get-got/discord-downloader-go.svg?branch=master" />
</a>
<a href="https://goreportcard.com/report/github.com/get-got/discord-downloader-go" alt="Go Report Card">
    <img src="https://goreportcard.com/badge/github.com/get-got/discord-downloader-go" />
</a>
<br>
<a href="https://discord.gg/6Z6FJZVaDV">
    <img src="https://img.shields.io/discord/780985109608005703?logo=discord"alt="Join the Discord">
</a>

<a href="https://github.com/get-got/discord-downloader-go/releases/latest">
    <b>DOWNLOAD LATEST RELEASE BUILDS</b>
</a>

This project is a fork of Seklfreak's discord-image-downloader-go

For list of differences and why I made an independent project, see below.

This is a Discord bot program to download files posted in specified Discord channels to local folders. It can fetch highest possible quality files from various sources (listed below), aside from downloading any file directly linked or attached. See Features below for full list.

Need help? Have suggestions? Join the support & discussion Discord server for this project!

WARNING: Discord does not allow Automated User Accounts (Self-Bots)

Read more in Discord Trust & Safety Team's Official Statement...

NOTE: This only applies to real User Accounts, not Bot users. This program currently works for either.

Getting Started (Basic Setup)

Obviously, download the bot program (or compile yourself with Go) and put it in a folder of your choice.

You can either create a settings.json following the examples & variables listed below, or have the program create a default file (if it is missing when you run the program, it will make one, and ask you if you want to enter in basic info for the new file). Ensure you follow proper JSON syntax to avoid any unexpected errors.

For Credentials...

• If using a Bot User, enter the token into the "token" setting. Remove the lines for "username" and "password" or leave blank (""). To create a Bot User, go to discord.com/developers/applications and create a New Application. Once created, go to Bot and create. The token can be found on the Bot page. To invite to your server(s), go to OAuth2 and check "bot", copy the url, paste into browser and follow prompts for adding to server(s).
• If using a Real User (Self-Bot), fill out the "username" and "password" settings. Remove the line for "token" or leave blank ("").
• If using a Real User (Self-Bot) with 2FA (Two-Factor Authentication), enter the token into the "token" setting. Remove the lines for "username" and "password" or leave blank (""). Token can be found from Developer Tools in browser under localStorage.token or in the Discord client Ctrl+Shift+I (Windows)/Cmd+Option+I (Mac) under Application → Local Storage → https://discordapp.com → "token".

Bot Permissions in Channels/Servers

• In order to perform basic downloading functions, the bot will need Read Message permissions in the server(s) of your designated channel(s).
• In order to respond to commands, the bot will need Send Message permissions in the server(s) of your designated channel(s). If executing commands via an Admin Channel, the bot will only need Send Message permissions for that channel, and that permission will not be required for the source channel.
• In order to process history commands, the bot will need Read Message History permissions in the server(s) of your designated channel(s).

How to Find Discord ID's

• Discord Developer Mode: Enable Developer Mode in Discord settings under Appearance.
• Finding Channel ID: Enable Discord Developer Mode (see above), right click on the channel and Copy ID.
• Finding User ID: Enable Discord Developer Mode (see above), right click on the user and Copy ID.
• Finding Emoji ID: Enable Discord Developer Mode (see above), right click on the emoji and Copy ID.

Features

• Supported File Downloading:
  • Discord File Attachments
  • Direct Links to Files
  • Twitter (requires API key, see config section)
  • Instagram
  • Facebook Videos
  • Imgur (Single Posts & Albums)
  • Flickr (requires API key, see config section)
  • Google Drive (requires API Credentials, see config section)
  • Tistory
  • Streamable
  • Gfycat
• Commands:
  • Help (help - Alias: commands)
  • Ping (ping - Alias: test)
  • Status: Get an output of the current status of the bot (status - Alias: info)
  • Stats: Have the bot dump stats (stats)
  • [Must be Bot or Server Admin] History: Process all old messages in channel (history - Aliases: catalog, cache)
  • [Must be Bot Admin] Exit (nice for process managers like pm2 for instant reload) (exit - Aliases: reload, kill)

  Differences from Seklfreak's discord-image-downloader-go & Why I made this

  • Go 1.15 rather than 1.13
  • discordgo 0.22.0 rather than 0.16.1
  • Implements dgrouter for commands
  • Configuration is JSON-based rather than ini to allow more elaborate settings and better organization. With this came many features such as channel-specific settings.
  • Channel-specific control of downloaded filetypes / content types (considers things like .mov as videos as well, rather than ignore them), Optional dividing of content types into separate folders.
  • (Optional) Reactions upon download success.
  • (Optional) Discord messages upon encountered errors.
  • Extensive bot status/presence customization.
  • Consistent Log Formatting, Color-Coded Logging
  • Somewhat different organization than original project; initially created from scratch then components ported over.
  • Added Download Support for: Facebook Videos
  • Fixed Compatability Issue with xurls that required people to edit the project, regarding xurls.Strict.FindAllString. The issue was due to some people having xurls v2 installed while the projects go.mod required v1.1; changing go.mod to require v2 specifically seems to be the correct fix.

  I've been a user of Seklfreak's project since ~2018 and it's been great for my uses, but there were certain aspects I wanted to expand upon, one of those being customization of channel configuration, and other features like message reactions upon success, differently formatted statuses, etc. If some aspects are rudimentary or messy, please make a pull request, as this is my first project using Go and I've learned everything from observation & Stack Overflow.

  History Cataloging Guide

  This guide is to show you how to make the bot go through all old messages in a channel and catalog them as though they were being sent right now, in order to download them all.

  You will need the Channel ID (see bottom of Setup) if attempting to catalog history from a specific channel or group of channels, within an admin channel.

  • <prefix>history to catalog the current channel the command is sent in (must be registered in channels in settings).
  • <prefix>history cancel to stop cataloging the current channel the command is sent in (must be registered in channels in settings).
  • <prefix>history <Channel ID(s)> to catalog specified channels from within a designated Admin Channel (must be registered in adminChannels in settings). You can do multiple channels per command if desired, separated by commas.
  • <prefix>history cancel <Channel ID(s)> to stop cataloging specified channels from within a designated Admin Channel (must be registered in adminChannels in settings). You can do multiple channels per command if desired, separated by commas.

  Settings / Configuration Guide

  I tried to make the configuration as user friendly as possible, though you still need to follow proper JSON syntax (watch those commas). All settings specified below labeled [DEFAULTS] will use default values if missing from the settings file, and those labeled [OPTIONAL] will not be used if missing from the settings file.

  Basic Settings Example

  The following example is for a Bot Application (using a token), bound to 1 channel.

  This setup exempts many options so they will use default values (see below). It shows the bare minimum required settings for the bot to function.

  When initially launching the bot it will create a default settings file if you do not create your own settings.json beforehand.

  Example - Barebones settings.json:

  {
      "credentials": {
          "token": "YOUR_TOKEN"
      },
      "channels": [
          {
              "channel": "DISCORD_CHANNEL_ID_TO_DOWNLOAD_FROM",
              "destination": "FOLDER_LOCATION_TO_DOWNLOAD_TO"
          }
      ]
  }
  

  Example - Selfbot settings.json:

  {
      "credentials": {
          "email": "REPLACE_WITH_YOUR_EMAIL",
          "password": "REPLACE_WITH_YOUR_PASSWORD"
      },
      "scanOwnMessages": true,
      "presenceEnabled": false,
      "channels": [
          {
              "channel": "DISCORD_CHANNEL_ID_TO_DOWNLOAD_FROM",
              "destination": "FOLDER_LOCATION_TO_DOWNLOAD_TO",
              "allowCommands": false,
              "errorMessages": false,
              "reactWhenDownloaded": false
          }
      ]
  }
  

  Example - Advanced settings.json:

  {
      "credentials": {
          "token": "YOUR_TOKEN",
          "twitterAccessToken": "",
          "twitterAccessTokenSecret": "",
          "twitterConsumerKey": "",
          "twitterConsumerSecret": ""
      },
      "admins": [ "YOUR_DISCORD_USER_ID", "YOUR_FRIENDS_DISCORD_USER_ID" ],
      "adminChannels": [
          {
              "channel": "CHANNEL_ID_FOR_ADMIN_CONTROL"
          }
      ],
      "debugOutput": true,
      "commandPrefix": "downloader_",
      "allowSkipping": true,
      "filterDuplicates": true,
      "filterDuplicatesThreshold": 75,
      "downloadRetryMax": 5,
      "downloadTimeout": 120,
      "githubUpdateChecking": true,
      "presenceStatus": "dnd",
      "presenceType": 3,
      "presenceOverwrite": "{{count}} files",
      "filenameDateFormat": "2006.01.02-15.04.05_",
      "embedColor": "#29BEB0",
      "inflateCount": 123456,
      "channels": [
          {
              "channel": "THIS_CHANNEL_DOWNLOADS_EVERYTHING",
              "destination": "EVERYTHING",
              "overwriteEmbedColor": "#FF0000",
              "userBlacklist": [ "USER_ID_FOR_PERSON_I_DONT_LIKE" ],
              "divideFoldersByType": false,
              "saveImages": true,
              "saveVideos": true,
              "saveAudioFiles": true,
              "saveTextFiles": true,
              "saveOtherFiles": true,
              "savePossibleDuplicates": true,
              "extensionBlacklist": [
                  ".htm",
                  ".html",
                  ".php"
              ]
          },
          {
              "channel": "THIS_CHANNEL_ONLY_DOWNLOADS_MEDIA",
              "destination": "media",
              "overwriteAllowSkipping": false,
              "saveImages": true,
              "saveVideos": true,
              "saveAudioFiles": true,
              "saveTextFiles": false,
              "saveOtherFiles": false
          },
          {
              "channel": "THIS_CHANNEL_ONLY_DOWNLOADS_MEDIA_FROM_ME_EXCEPT_DISCORD_FILES",
              "destination": "media2",
              "usersAllWhitelisted": false,
              "userWhitelist": [ "MY_USER_ID" ],
              "saveImages": true,
              "saveVideos": true,
              "saveAudioFiles": true,
              "saveTextFiles": false,
              "saveOtherFiles": false,
              "domainBlacklist": [ "cdn.discordapp.com" ]
          },
          {
              "channel": "THIS_CHANNEL_IS_STEALTHY",
              "destination": "stealthy files",
              "allowCommands": false,
              "errorMessages": false,
              "updatePresence": false,
              "reactWhenDownloaded": false
          },
          {
              "channels": [ "CHANNEL_1", "CHANNEL_2", "CHANNEL_3", "CHANNEL_4", "CHANNEL_5" ],
              "destination": "lots of stuff",
              "allowCommands": false,
              "errorMessages": false,
              "updatePresence": false
          }
      ]
  }
  

  All JSON settings follow camelCase format.

  List of Settings

  • credentials [key/value object]
    • token [string]
      • Required for Bot Login or User Login with 2FA, don't include if using User Login without 2FA.
    • email [string]
      • Required for User Login without 2FA, don't include if using Bot Login.
    • password [string]
      • Required for User Login without 2FA, don't include if using Bot Login.
    • [OPTIONAL] twitterAccessToken [string]
      • Won't use Twitter API for fetching media from tweets if credentials are missing.
    • [OPTIONAL] twitterAccessTokenSecret [string]
      • Won't use Twitter API for fetching media from tweets if credentials are missing.
    • [OPTIONAL] twitterConsumerKey [string]
      • Won't use Twitter API for fetching media from tweets if credentials are missing.
    • [OPTIONAL] twitterConsumerSecret [string]
      • Won't use Twitter API for fetching media from tweets if credentials are missing.
    • [OPTIONAL] flickrApiKey [string]
      • Won't use Flickr API for fetching media from posts/albums if credentials are missing.
    • [OPTIONAL] googleDriveCredentialsJSON [string]
      • Path for Google Drive API credentials JSON file.
      • Won't use Google Drive API for fetching files if credentials are missing.
  • [OPTIONAL] admins [array of strings]
    • Array of User ID strings for users allowed to use admin commands
  • [OPTIONAL] adminChannels [array of key/value objects]
    • channel [string]
  • [DEFAULTS] debugOutput [bool]
    • Default: false
    • Output debugging information.
  • [DEFAULTS] commandPrefix [string]
    • Default: "ddg "
  • [DEFAULTS] allowSkipping [bool]
    • Default: true
    • Allow scanning for keywords to skip content downloading.
    • "skip", "ignore", "don't save", "no save"
  • [DEFAULTS] scanOwnMessages [bool]
    • Default: false
    • Scans the bots own messages for content to download, only useful if using as a selfbot.
  • [DEFAULTS] filterDuplicateImages [bool]
    • Default: false
    • Experimental feature to filter out images that are too similar to other cached images.
    • Caching of image data is stored via a database file; it will not read all pre-existing images.
  • [DEFAULTS] filterDuplicateImagesThreshold [float64]
    • Default: 100.0
    • Threshold for what the bot considers too similar of an image comparison score. Lower = more similar (lowest is around -109.7), Higher = less similar (does not really have a maximum, would require your own testing).
  • [DEFAULTS] downloadRetryMax [int]
    • Default: 3
  • [DEFAULTS] downloadTimeout [int]
    • Default: 60
  • [DEFAULTS] githubUpdateChecking [bool]
    • Default: true
    • Check for updates from this repo.
  • [DEFAULTS] presenceEnabled [bool]
    • Default: true
  • [DEFAULTS] presenceStatus [string]
    • Default: "idle"
    • Presence status type.
    • "online", "idle", "dnd", "invisible", "offline"
  • [DEFAULTS] presenceType [int]
    • Default: 0
    • Presence label type. ("Playing <activity>", "Listening to <activity>", etc)
    • Game = 0, Streaming = 1, Listening = 2, Watching = 3, Custom = 4
      • If Bot User, Streaming & Custom won't work properly.
  • [OPTIONAL] presenceOverwrite [string]
    • Unused by Default
    • Replace counter status with custom string.
    • Embedded Placeholders:
      • {{dgVersion}}: discord-go version
      • {{ddgVersion}}: Project version
      • {{apiVersion}}: Discord API version
      • {{countNoCommas}}: Raw total count of downloads (without comma formatting)
      • {{count}}: Raw total count of downloads
      • {{countShort}}: Shortened total count of downloads
      • {{numGuilds}}: Number of guilds bot is a member of
      • {{numChannels}}: Number of bound channels
      • {{numAdminChannels}}: Number of admin channels
      • {{numAdmins}}: Number of designated admins
      • {{timeSavedShort}}: Last save time formatted as 3:04pm
      • {{timeSavedShortTZ}}: Last save time formatted as 3:04pm MST
      • {{timeSavedMid}}: Last save time formatted as 3:04pm MST 1/2/2006
      • {{timeSavedLong}}: Last save time formatted as 3:04:05pm MST - January 1, 2006
      • {{timeSavedShort24}}: Last save time formatted as 15:04
      • {{timeSavedShortTZ24}}: Last save time formatted as 15:04 MST
      • {{timeSavedMid24}}: Last save time formatted as 15:04 MST 2/1/2006
      • {{timeSavedLong24}}: Last save time formatted as 15:04:05 MST - 1 January, 2006
      • {{timeNowShort}}: Current time formatted as 3:04pm
      • {{timeNowShortTZ}}: Current time formatted as 3:04pm MST
      • {{timeNowMid}}: Current time formatted as 3:04pm MST 1/2/2006
      • {{timeNowLong}}: Current time formatted as 3:04:05pm MST - January 1, 2006
      • {{timeNowShort24}}: Current time formatted as 15:04
      • {{timeNowShortTZ24}}: Current time formatted as 15:04 MST
      • {{timeNowMid24}}: Current time formatted as 15:04 MST 2/1/2006
      • {{timeNowLong24}}: Current time formatted as 15:04:05 MST - 1 January, 2006
      • {{uptime}}: Shortened duration of bot uptime
  • [OPTIONAL] presenceOverwriteDetails [string]
    • Unused by Default
    • Replace counter status details with custom string (only works for User, not Bot).
    • Embedded Placeholders:
      • See presenceOverwrite above...
  • [OPTIONAL] presenceOverwriteState [string]
    • Unused by Default
    • Replace counter status state with custom string (only works for User, not Bot).
    • Embedded Placeholders:
      • See presenceOverwrite above...
  • [DEFAULTS] filenameDateFormat [string]
    • Default: "2006-01-02_15-04-05 "
    • See this Stack Overflow post regarding Golang date formatting.
  • [OPTIONAL] embedColor [string]
    • Unused by Default
    • Supports random/rand, role/user, or RGB in hex or int format (ex: #FF0000 or 16711680).
  • [OPTIONAL] inflateCount [int]
    • Unused by Default
    • Inflates the count of total files downloaded by the bot. I only added this for my own personal use to represent an accurate total amount of files downloaded by previous bots I used.
  • channels [array of key/value objects]
    • [THIS OR BELOW] channel [string]
      • Channel ID to monitor.
    • [THIS OR ABOVE] channels [array of strings]
      • Channel IDs to monitor, for if you want the same configuration for multiple channels.
    • destination [string]
      • Folder path for saving files, can be full path or local subfolder.
    • [DEFAULTS] enabled [bool]
      • Default: true
      • Toggles bot functionality for channel.
    • [DEFAULTS] allowCommands [bool]
      • Default: true
      • Allow use of commands like ping, help, etc.
    • [DEFAULTS] errorMessages [bool]
      • Default: true
      • Send response messages when downloads fail or other download-related errors are encountered.
    • [DEFAULTS] scanEdits [bool]
      • Default: true
      • Check edits for un-downloaded media.
    • [DEFAULTS] updatePresence [bool]
      • Default: true
      • Update Discord Presence when download succeeds within this channel.
    • [DEFAULTS] reactWhenDownloaded [bool]
      • Default: true
      • Confirmation reaction that file(s) successfully downloaded.
    • [OPTIONAL] reactWhenDownloadedEmoji [string]
      • Unused by Default
      • Uses specified emoji rather than random server emojis. Simply pasting a standard emoji will work, for custom Discord emojis use "name:ID" format.
    • [OPTIONAL] overwriteFilenameDateFormat [string]
      • Unused by Default
      • Overwrites the global setting filenameDateFormat (see above)
      • See this Stack Overflow post regarding Golang date formatting.
    • [OPTIONAL] overwriteAllowSkipping [bool]
      • Unused by Default
      • Allow scanning for keywords to skip content downloading.
      • "skip", "ignore", "don't save", "no save"
    • [OPTIONAL] overwriteEmbedColor [string]
      • Unused by Default
      • Supports random/rand, role/user, or RGB in hex or int format (ex: #FF0000 or 16711680).
    • [DEFAULTS] usersAllWhitelisted [bool]
      • Default: true
      • Allow messages from all users to be handled. Set to false if you wish to use userWhitelist to only permit specific users messages to be handled.
    • [OPTIONAL] userWhitelist [array of strings]
      • Use with usersAllWhitelisted as false to only permit specific users to have their messages handled by the bot. Only accepts User ID's in the array.
    • [OPTIONAL] userBlacklist [array of strings]
      • Use with usersAllWhitelisted as the default true to block certain users messages from being handled by the bot. Only accepts User ID's in the array.
    • [DEFAULTS] divideFoldersByServer [bool]
      • Default: false
      • Separate files into subfolders by server of origin (e.g. "My Server", "My Friends Server")
    • [DEFAULTS] divideFoldersByChannel [bool]
      • Default: false
      • Separate files into subfolders by channel of origin (e.g. "my-channel", "my-other-channel")
    • [DEFAULTS] divideFoldersByUser [bool]
      • Default: false
      • Separate files into subfolders by user who sent (e.g. "Me#1234", "My Friend#0000")
    • [DEFAULTS] divideFoldersByType [bool]
      • Default: true
      • Separate files into subfolders by type (e.g. "images", "video", "audio", "text", "other")
    • [DEFAULTS] saveImages [bool]
      • Default: true
    • [DEFAULTS] saveVideos [bool]
      • Default: true
    • [DEFAULTS] saveAudioFiles [bool]
      • Default: false
    • [DEFAULTS] saveTextFiles [bool]
      • Default: false
    • [DEFAULTS] saveOtherFiles [bool]
      • Default: false
    • [DEFAULTS] savePossibleDuplicates [bool]
      • Default: true
    • [DEFAULTS] extensionBlacklist [array of strings]
      • Default: [ ".htm", ".html", ".php", ".exe", ".dll", ".bin", ".cmd", ".sh", ".py", ".jar" ]
      • Ignores files containing specified extensions. Ensure you use proper formatting.
    • [OPTIONAL] domainBlacklist [array of strings]
      • Ignores files from specified domains. Ensure you use proper formatting.
    • [OPTIONAL] saveAllLinksToFile [string]
      • Saves all sent links to file, does not account for any filetypes or duplicates, it just simply appends every raw link sent in the channel to the specified file.

  Info for Developers

  • I'm a complete amateur with Golang. If anything's bad please make a pull request.
  • Versioning is [MAJOR].[MINOR].[PATCH]
  • I try to be consistent with annotation but it's not perfect.
  • Logging generally follows certain standards and patterns with formatting and color-coding.