Author here. ConvNetJS is a project I maintain on a side for fun. A part of my motivation was that I wanted to make these algorithms and techniques more approachable, and easier to understand, play with and apply. One issue right now is that I think I plunged into development and wrote a whole bunch of code and visualizations without fully supporting it with the necessary tutorials for a complete beginner. I'm hoping to change that over the next few weeks. Sometimes it's a little hard to juggle this with research, and all the things I'm actually supposed to be doing.
I'd be happy to answer questions or get feedback on the project!
This is great! Although if your goal is to make this more approachable, may I suggest that you include a diagram in your documentation, explaining how the layers look and how they are interconnected? I studied ANNs in university, but after 10 years, even I have trouble recalling some details.
Even after taking several machine learning classes and learning about neural networks several times, I still don't have a good intuition on how these network works in practice. Being able to visualize how learning algorithm evolves is super helpful. Awesome work karpathy!
What would you like to know? I assume you've already seen the hinton coursera class[3]?
Disclaimer: author.
I put some hints in to practice here in my framework[1].
This is generally applicable to any neural network.
My live talk that I gave just recently[2] goes in to practical ways of training neural nets without a lot of the math behind it (more just: do this and you get this effect)
Coming from the opposite direction, I understand neural networks quite well, including deep learning, but am lacking on many other topics such as structured SVMs, conditional random fields, etc. Any recommendations for me?
Sure. Have you looked at the PGM class on coursera for CRFs? The slides are a great condensed version of koller's book on PGMs. NLP is a really easy context they are used in. With that, I would recommend the stanford NLP coursera class (the first one from a few years back) that covers viterbi, CRFS, and general sequential models.I know I'm recommending a lot of MOOC content here, but I would add here that I think they provide a great overview from which to break in to the papers/other literature on the topic.
For SVMs, outside of the typical research you'd do on wikipedia, I unfortunately haven't had much specific experience with SVMs. I used them quite a bit a few years back for relation extraction and other algorithms, but I'm mainly from an NLP background in that context.
Thanks! This is helpful. I'll check both out courses.
The recommendation for PGM sounds promising. I am familiar with Bayes theorems (through the first offering of the AI course on what later became Udacity).
I had signed up for NLP on Coursera but dropped in it the middle since I was already familiar with the contents till that point (having read most parts of both Christopher Manning's and Dan Jurafsky's books already). I'll check the materials for the rest of it.
I'd be happy to answer questions or get feedback on the project!