A giant star is a star much larger than a main sequence star, with a radius on the order that of a solar system planet's orbit (e.g., an astronomical unit). After their main sequence phase, stars go through phases in which they become giants, e.g., the red-giant branch (RGB), horizontal branch (HB), and asymptotic giant branch (AGB).
The term originally applied to especially bright stars, i.e., with a large luminosity (absolute magnitude). At a given temperature, a star's brightness is directly related to the area facing us (the square of the radius), and many of the brightest stars are indeed large, known as bright giants or for even brighter, supergiants. But merely being bright also includes massive main-sequence stars such as O-type stars, which are generally not (I don't think) termed giants. The general term dwarf star actually applies to any star that is not a giant (e.g., the Sun), though often "dwarf" is used with a qualifier to signify a more specific meaning (white dwarf, red dwarf, brown dwarf).
Stars grow to be giants when their luminosity is high and their mass is low, i.e., a lower mass than that of a main-sequence star of similar luminosity, which would often be a (main-sequence) B-type star. Luminosity (energy production) is much higher in some post main sequence phases due to larger volumes supporting the conditions to produce fusion, and/or the occurrence of additional types of fusion due to the necessary density, temperature and fuel. The low mass allows the star to "puff up", and with the much larger surface, the temperature at which the photosphere radiates energy equal to what is passed to it from inside, is cooler, resulting in a more-reddish color. (A main-sequence star with a similar luminosity has its larger mass's gravity keeping the star more compact.) The inflation of the star is due to the balance of forces/mass, including the heat of the extensive fusion, and is a bit non-linear because gas pushed further from center, in turn has less pull from the star's gravity, which follows an inverse square law. A major factor is the outward radiation pressure: radiative transfer is a net outward movement of photons, producing a net outward force.
Giant stars may be of different colors, red (red giants) being most common, but can be blue (blue giants) or yellow (yellow giants), and the term white giant is sometimes used for stars between the two in temperature. The largest are red: a smaller giant (subgiant) may simply be a less bright star, or may find its balance at a somewhat smaller size, their luminosity producing a higher temperature, making them less red. These alternate colors can occur during different post-main-sequence phases, with other characteristics of the star being additional deciding factors, including the star's mass and metallicity.