“What dangers he might meet he could not know, so he selected three spells of general application: the Excellent Prismatic Spray, Phandaal’s Mantle of Stealth, and the Spell of the Slow Hour.”
All magic comes from other planes,1) but it varies in nature. The act of casting a spell is an act of calling the power of another plane.
Magic-user magic, or “arcane magic”, comes from the whole gamut of planes. It is formulaic and arcane, and is learned through study.
Clerical magic, or “divine magic”, comes from divine ritual, servants of deities, and deities themselves. Upsetting a deity or their servants may lead a character to lose access to clerical magic above the 2nd level. The character may be granted spells beyond that for purposes of atonement, or whatever the granting entities deem appropriate.
Druidic magic comes from the inner planes, especially the elemental, para-elemental, and quasi-elemental planes. If a druid acts anathema to their class, such as destroying woodlands or harming innocent woodland creatures, they are apt to sever their connection to their magic ability, losing their ability to cast some or all spells until atonement is made.
Paladin magic is similar to clerical magic in results, but does not come from divine ritual, deities, or their servants. Paladins, as avatars of their alignment, draw their power directly from the Seven Heavens. They can lose spellcasting ability (and indeed, all their paladin powers) by knowingly performing a chaotic or evil act. Chaotic acts can be atoned for, but evil acts cannot.
Illusionist magic, or “phantasmal magic” as it is sometimes called, is sourced from all the planes, as magic-user magic. However, the illusionist’s study and use of magic differs. Their spell books and scrolls are written in a secret, arcane tongue necessary for illusionistic conjuring.2)