Re: [CentOS] strange system outage

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: [CentOS] strange system outage

fred smith
On Wed, May 10, 2017 at 04:35:07PM -0400, Larry Martell wrote:

> On Wed, May 10, 2017 at 4:18 PM, Jonathan Billings <[hidden email]> wrote:
> > On Wed, May 10, 2017 at 03:19:05PM -0400, Larry Martell wrote:
> >> > How are you starting this daemon?
> >>
> >> I am using code something like this: https://gist.github.com/slor/5946334.
> >
> > Oh, I was assuming that since you called it a daemon, it was actually
> > something started automatically on boot, instead of something you
> > manually started and it daemonized.
>
> So on all the other systems that this is deployed on it is started
> automatically at boot and it's controlled with service or systemctl.
> On this system because I do not have remote access and I do not have a
> very competent set of hands over there it was never set up to run as a
> service. So a user manually starts it by running the daemon script.
>
> This had been running on the machine like that for 8 months. The
> nightly crash just started on April 21 and has happened every day
> since.
>
> > If it is dying, are you logged in when its running?
>
> I am not 100% sure, but I think the user sus, starts the daemon
> process and then exits the su.
>
> > does it require you to be connected when its running?
>
> No. The underlying script polls dirs looking for files, and loads data
> into a db.
>
> > Maybe you should run it in a
> > screen/tmux instead of daemonizing, so you can see stderr/stdout?
>
> stdout and stderr are written to a file and when the daemon gets
> killed or dies or whatever happens to it, the output in the file
> abruptly stops, just like what I showed in the messages file.

Could you enable core dumps (ulimit -c unlimited) then after it
has died, check for core files?

--
---- Fred Smith -- [hidden email] -----------------------------
                      The eyes of the Lord are everywhere,
                    keeping watch on the wicked and the good.
----------------------------- Proverbs 15:3 (niv) -----------------------------
_______________________________________________
CentOS mailing list
[hidden email]
https://lists.centos.org/mailman/listinfo/centos