The report motivates, from different viewpoints, an increased level of autonomy in unmanned airborne surveillance systems equipped with infrared and video sensors. In order to raise the level of autonomy in these systems, it is necessary to take into account the uncertainty associated with the percepts of a cluttered and rapidly changing environment. The proposed signal processing framework for autonomous UAV surveillance combines image processing, such as ground target detection and landmark recognition, with planning and management of multiple surveillance requests, sensor and platform control, and sensor data fusion. A client/server model is suggested for handling on-line adaptable surveillance missions. Our working hypothesis is that integration of the detection-tracking-classification chain with spatial awareness makes possible intelligent autonomous data acquisition by means of active sensor control and route planning. A central part of the framework is a surveillance scene representation, suitable for target tracking, geolocation, and sensor data fusion involving multiple platforms. The representation, based on Simultaneous Localization and Mapping, SLAM, takes into account uncertainties associated with sensor data, platform navigation, and prior knowledge. The ability of the representation to handle distributed data fusion and cooperative information gathering using multiple platforms makes it ideally suited for establishing a "common picture".