simple log management and viewing for your servers

As a guy who develops, deploys and maintains webapps, I had to login to my servers and tail the logs to see issues too many times. It’s a real pain, And anybody who maintains any servers knows this.

I’ve recently found a bunch of very good apps, which make this job very pleasant: PaperTrail is an awesome app which makes it very simple to setup a logging daemon and view all your logs (from all your servers) on their website, It’s a very neat implementation. But, you might not want to send your logs to other apps as they usually have sensitive information. PaperTrail

logstash is another awesome open source implementation for log management. With logstash, you can setup a small central server which collects all your logs and allows you to access them through a variety of interfaces. Another advantage of logstash is that the logs stay on your server under your control, plus it’s open source. The only downside is the one time setup, which, is not that hard. It is very versatile in ways it allows you access to your logs. LogStash

If none of them seem to be your thing, here is a small script which I use to tail remote log files. It runs tail -f over an ssh connection. It’s very simple to setup and use. Once you set it up, you can just cd into your application directory and run rt and it will start tailing your log files instantly. If you have any improvements you can fork this gist and update it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
#~/.scripts/rt
#!/usr/bin/env ruby
require 'rubygems'
require 'yaml'
require 'erb'
ConfigFilePath = File.expand_path("~/.remote-tail.yml")
#will be written out to ConfigFilePath if not present
SampleConfig=<<EOS
---
defaults: &defaults
host: c3
foonginx:
file: /var/log/nginx/*.log
barapp:
host: foo@bar.com
file: /var/www/apps/railsfooapp/shared/log/*.log
EOS
Usage=<<EOS
Usage:
1. cd to into the directory whose name is the same as the name of the config and run
rt
2. rt <name of the app>
3. rt <host> <file>
EOS
def tail(opts)
cmd = "ssh #{opts['host']} 'tail -f #{opts['file']}'"
puts "running: '#{cmd}'"
system(cmd)
end
def config(app)
puts "using app:#{app}"
config = YAML::load_file ConfigFilePath
return config[app] if config[app]
puts "app:#{app} not found in #{ConfigFilePath}"
puts Usage
exit 2
end
def setup
return if File.exist? ConfigFilePath
puts "creating a sample config at: #{ConfigFilePath}"
File.open(ConfigFilePath, 'w') do |f|
f.print SampleConfig
end
end
def init
setup
case ARGV.length
when 0
#usage:
#cd to the app root directory, usually this would be the name with which you
#setup the configuration and run
#$ rt
tail config(File.basename(Dir.pwd))
when 1
#usage:
#from any directory
#$ rt <name of the app>
tail config(ARGV.first)
when 2
#usage:
#from any directory
#$ rt <host> <file>
tail :host => ARGV.first, :file => ARGV.last
else
puts "Invalid number of arguments"
puts Usage
exit 1
end
end
init

I am currently working on Zammu which makes Automatic Deployment of static websites to Github Pages very easy. I would love to get your feedback on it, Use the invitation code KHAJA