Menut et al. show that microgels have different elastic properties based on their particle size. In compression, both colloidal and granular macrogels behave similiarly due to tight packing densities. Here osmotic events are limited to elastic contributions by the gel matrix, and solvent expulsion - neither of which are effected by particle size. However, microgel shear response is size dependent. Brownian motion determines the elastic properties of colloidal particles while solvent-polymer interactions govern the response of granular systems. This result was verified by testing chemically distinct microgels. In colloidal systems, Brownian movement, allows the particles to rapidly deform and move independent of system chemistry, while in granular systems, shear response varied widely depending on the chemical nature of the microgel.