IMHO "programming language" would refer specifically to the grammar, type system, semantics, keywords, etc. and "compiler" would refer to the implementation that takes something written in that representation and outputs some type of executable. A pedantic distinction but probably a useful one, especially given how some languages have multiple implementations of their standard.
The Programming Languages course I took in college had us implement a toy subset of Ruby targeting the Lua VM. Easily the most interesting and useful class that I had the pleasure of experiencing; it completely changed how I think about languages. I think everybody should have exposure to the basic principles of compilers like syntax trees, flow analysis, calling conventions, and so forth (if not necessarily the more complicated stuff like operational semantics).
There are two neat things about compilers and interpreters:
1. They are one of the most researched areas of CS with a large amount of educational materials, tools, and community knowledge to help.
2. They map to any problem involving transforming an input into an output (hell, any problem transforming an input to an output is a compiler or interpreter).
I'd argue the idea of an "interpreter" is one of the most foundational concepts of CS.
It sounds really basic, but the idea of being handed a description, and doing/creating something based on that is everywhere. It is really quite beautiful.
It comes up often too. A few month ago I wanted to make a part of the code I was working on more declarative, so I had it take a javascript object that described a layout, and make actions depending on that object. That's a form of interpreter in a way.
To me it was the recursive mapping over a closed set of types, whatever you do you should end up with an element of your type/grammar. That and the graph walk space of the AST.
The closed set of types describes how each pass in a compiler pipeline tends to build an intermediate representation of some sort, analyze/modify it, and then output it to the next stage. The IR is closed in that it is not open for everyone to extend willy-nilly. It is this property that makes it even possible to analyze and manipulate in a sound manner.
In a sense, compilers literally build a bunch of linguistic 'worlds' within their pipeline, each one emphasizing something different.
The Programming Languages course I took in college had us implement a toy subset of Ruby targeting the Lua VM. Easily the most interesting and useful class that I had the pleasure of experiencing; it completely changed how I think about languages. I think everybody should have exposure to the basic principles of compilers like syntax trees, flow analysis, calling conventions, and so forth (if not necessarily the more complicated stuff like operational semantics).