Optimizing Filters for Speed

Components and Render Speed

Bitmap-based components (Blur, Motion Blur, Sharpen and High Pass) with any slow component (like Worley-based noise) as a source.

Worley-based noises (any Noise except Perlin, which is very fast).
Bitmap-based components (see above) in general, because they need to prepare a source bitmap before rendering themselves.

Refraction, especially with a slow component as a source. Refraction takes 3 samples of the source input.
Any Noise (or any component based on Noise such as Noise Gradient, Noise Distortion or Noise Curve) with high Detail and Roughness (they are interdependent) especially with a custom curve as a profile.
Any Noise with a custom curve as a profile -- the curve is sampled once for each noise octave (for Details = 10 you have 10 samples of the curve per one noise sample).

Any component that generates edges for anti-aliasing (Patterns, Worley noises, some of the adjustments). But since anti-aliasing is done in a separate pass, that shouldn't impact speed too much.

You can use multiple duplicates of the Image and Selection component, this won't affect rendering speed.
It is not recommended to duplicate other components. The renderer has an optimization called "sample cache" that can save a significant amount of rendering time in situations when a single component has multiple outbound connections. This speedup will not occur when using duplicates instead of connections.
If you have multiple outbound connections going to different distorters (e.g. Offset and Noise Distortion), which also have multiple outbound connections, try inserting a disabled Invert before each distorter branch. This can help preventing sample cache flushing.