Briefly, a component is the main part of the page. For example, com_content is the component that runs the section and category blogs and articles. com_contact displays contact information. com_weblinks shows your web links (hehe! my examples look silly, the names are so intuitive.)
Also, your menu items are always linking to a component because every Joomla! page is "primarily" the component output.
Modules are those items that are typically around the outside of the component (doesn't have to display there, but typically, most of us think inside the box with design and that's where we place modules.) The most "important" modules are your menus. But, other core modules include the login module or the breadcrumbs or the latest news or most popular news. Modules typically display data, but are a secondary source of information on your page, compared to your component.
Plugins are cool little pieces of logic that do something that is not interactive with the site visitor. The plugin that I think is most useful is {loadposition userX} where you can actually place the output from a module (in the example I provided, the modules defined in position userx) right into your article. It's pretty nifty. Other plugins help with system activity or can be used to load videos or format articles in a certain way.
Typically, if you get an extension that has components, modules and plugins, you will need to install and configure each of them separately. It might seem a hassle, but I always look at it as an opportunity to see what software capability is really under the hood and consider how I'll use it.
Hope that helps!
Amy
