2 min readfrom Machine Learning

[P] I built a simple gpu-aware single-node job scheduler for researchers / students

[P] I built a simple gpu-aware single-node job scheduler for researchers / students
[P] I built a simple gpu-aware single-node job scheduler for researchers / students

(reposting in my main account because anonymous account cannot post here.)

Hi everyone!

I’m a research engineer from a small lab in Asia, and I wanted to share a small project I’ve been using daily for the past few months.

During paper prep and model development, I often end up running dozens (sometimes hundreds) of experiments. I found myself constantly checking whether GPUs were free, and even waking up at random hours just to launch the next job so my server wouldn’t sit idle. I got tired of that pretty quickly (and honestly, I was too lazy to keep writing one-off scripts for each setup), so I built a simple scheduling tool for myself.

It’s basically a lightweight scheduling engine for researchers:

  • Uses conda environments by default
  • Open a web UI, paste your command (same as terminal), choose how many GPUs you want, and hit submit
  • Supports batch queueing, so you can stack experiments and forget about them
  • Has live monitoring + built-in logging (view in browser or download)

Nothing fancy, just something that made my life way easier. Figured it might help others here too.

If you run a lot of experiments, I’d love for you to give it a try (and any feedback would be super helpful).

Github Link: https://github.com/gjamesgoenawan/ant-scheduler

submitted by /u/Zerokidcraft
[link] [comments]

Want to read more?

Check out the full article on the original site

View original article

Tagged with

#natural language processing for spreadsheets
#generative AI for data analysis
#Excel alternatives for data analysis
#rows.com