A brown dwarf (BD) is a star-like object which has insufficient mass to sustain fusion (the hydrogen-burning minimum mass aka HBMM) in the manner of main sequence stars ("stable fusion", i.e., of 1hydrogen), but with enough to trigger deuterium burning (the deuterium-burning minimum mass aka DBMM). This is roughly (and standardized as) 13 to 75 Jupiter masses. They can have an effective temperatures (Teff) ranging from 250 K to 3000 K, and any of the spectral classes, M-type star (M dwarf), L-type star (L dwarf), T-type star (T dwarf), or Y-type star (Y dwarf). They cool over the course of their life, so their classification evolves, which presents a mass/age degeneracy: that a young brown dwarf within the lower end of the mass range has a Teff (and spectral class) identical to that of more massive brown dwarfs that are sufficiently older. (Note that there isn't complete consistency regarding whether the term star is meant to include brown dwarfs, likely depending upon the context, the subdiscipline, and/or the writer.)
The coolest main sequence stars are within the hotter end of the brown-dwarf range, and the classifications M and L can be hydrogen burning stars. Also, many planets are warmer than 250 K. Consequently, even with the spectral class and absolute magnitude determined, its mass is needed to determine whether the object is a planet, a brown dwarf, or a star. Thus, when candidates (substellar objects of some kind) are found at distant orbits from a star, so mass-determination is possible as well as spectrography from direct imaging, characteristics of such bodies can be collected, which, in turn, assist in classifying similar objects found nearer the stars, e.g., by transits. Modeling such bodies has the extra complication of the possibility of clouds in the atmosphere.
There is a similarity between brown dwarfs and giant planets, such as Jupiter; both are basically hydrogen and helium, as are main sequence stars. Both are presumed to compress their hydrogen at the core to the point that it is electron degenerate matter (metallic hydrogen) without achieving a sufficient temperature/density combination to ignite the hydrogen. But star formation and planet formation theories are decidedly different: the concept of brown dwarfs grew from the idea that there would be occasions when the star formation process would not accumulate enough mass. Jupiter is presumed to have formed subsequent to the Sun, from the solar nebula and there are presumably differences between such a giant planet and such a failed star. Stars do often form in binary pairs and the fact that the object is orbiting a star is insufficient criteria.
The early-established observables used to identify brown dwarfs, as opposed to main sequence stars, were the presence of lithium and/or methane (Main sequence stars generally burn their lithium early, thus its presence is a sign that the star has never burned hydrogen). The first observed brown dwarf was identified as such in 1995. As of 2013, hundreds of brown dwarfs are known.
Brown dwarfs' rotation periods are generally hours, typical of a planet, rather than multiple days, typical of a star. They lack stars' phases of stellar wind, which reduces a star's angular momentum. Also, they grow smaller as they cool, boosting their rotation.
The mass criteria (13 to 75 Jupiter masses) is an imperfect indicator of whether the object burns hydrogen, deuterium, or nothing: this also depends on the constituents of the object (i.e., metallicity). It is thought that brown dwarfs actually can range from 10 to 90 Jupiter masses. The 13-75 range is at best suitable for quick or preliminary classification, a range within which brown dwarf is currently considered the most likely possibility.
The terms black dwarf, planetar and substar were also used for brown dwarfs in the 1960s/1970s before a consensus developed regarding the term.
(The term black dwarf is currently used for future white dwarfs, to describe them after they have cooled to a more-planet-like temperature.)