Ceramics are brittle, so imagine what porous ceramics are like. This weakness is often a major obstacle in many applications. The addition of a ductile phase such as a polymer to a porous ceramics can help overcome this intrinsic limitation. Yet, most studies so far have focused on the processing and characterization of dense composites, and to some extent to porous ceramic composites. Unidirectionality, with macropores arranged along a common direction, is another strategy to improve the strength of porous ceramics, while being beneficial for a wide range of other applications involved with mass, gas or species transport. Here we combine the two approaches and show a simple processing strategy to obtain highly porous, unidirectional ceramic/polymer composites. We infiltrated ice-templated porous zirconia scaffolds with a dilute solution of polymer (polycaprolactone or epoxy). After the evaporation of the solvent, porous ceramic composites with a porosity greater than 60% were obtained. We performed a complete mechanical characterization to assess the relative importance of the properties of the ceramic and polymer phases. Our results demonstrate that the addition of a ductile polymer (PCL) can increase both the strength and the toughness of the composites while maintaining a high porosity. However, the addition of a brittle polymer (epoxy) has seemingly no impact on the fracture properties. This approach could provide porous materials that are easier to handle for biomedical applications.