Timber blocks plank mining.

I have huge wisteria plank floor in my store in my realm. I added Timber and Efficiency V on my axe and it seems like these enchantments could be conflicting when mining planks. Before it was very fast to mine the floor if I needed any changes, but right now I can’t just run around chopping planks even with Haste II effect from beacon. When I try to mine it quickly mines the block but puts it back in a milisecond and I have to mine it slowly.