Tikzpicture and matrix

Tikzpicture and matrix

I want use tikzpicture for draw my matrix, U=[matrix], using this code
\begin{align*}
U&=
\begin{tikzpicture}[decoration=brace]
\matrix (m)[matrix of math nodes,left delimiter=[,right delimiter={]},ampersand replacement=\&] {
    \dfrac{2}{N}-1 \& \dfrac{2}{N} \& \cdots \& \dfrac{2}{N} \& \cdots\& \dfrac{2}{N} \\
    \vdots \& \vdots \& \ddots \& \cdots \& \vdots \& \vdots \\
    \,\,\,\,\,\dfrac{2}{N}\,\,\,\, \& \dfrac{2}{N} \& \dfrac{2}{N} \& \dfrac{2}{N}-1 \& \cdots \& \dfrac{2}{N} \\
    \vdots \& \vdots \& \vdots \& \vdots \& \ddots \& \vdots\\
    \dfrac{2}{N} \& \dfrac{2}{N} \& \cdots \& \dfrac{2}{N} \& \cdots  \& \dfrac{2}{N}-1\\
};
    \draw[decorate,transform canvas={xshift=-1.4em},thick] (m-3-1.south west) -- node[left=2pt] {$m$} (m-1-1.north west);
    \draw[decorate,transform canvas={yshift=0.5em},thick] (m-1-1.north west) -- node[above=2pt] {$m$} (m-1-4.north east);
\end{tikzpicture}
\end{align*}
but I can not get align U=. How I will be able to make this?