Formal and accountable security systems are rare in many of the fragile states where the international community provides assistance. In most cases, individuals and communities create their own security mechanisms or accept compromised and unaccountable security provided by non-state actors. This article examines how such alternative security mechanisms have evolved in communities in two fragile states, Colombia and Liberia. The authors conclude that the ability of both state and non-state actors to provide security hinges on perceptions of the legitimacy of those providing security among the population at large. Systems that involve community representatives in managing security mechanisms are critical to developing this legitimacy. Without attention to this key element, programmes for security sector reform are unlikely to succeed.