I would love it if everyone came to my party, but if I'm honest I really want those of you who want to be there. Because if you want to be there then you get it, you understand, you have a connection, a desire to be part of what I'm doing and celebrating.
August Burns Red isn't music for everyone, and if they start mangling their style in hopes of attracting more people... they'll most likely do the opposite.
The Puffy Shirt isn't for everyone, pirates I guess, but you'll lose the pirate crowd if you try a one-shirt-fits-all-tastes approach to puffy shirt making.
Create for the people who want your creation and two things will happen:
As they love it and live it they'll attract and invite people into the tribe,
As you create what only you create the way you create it and find ways to connect it with the world you'll find your tribe and your tribe will find you.
So, don't worry about connecting with the haters and the disinterested. Cast your vision, set out on mission and live your values through your art and you'll find a tribe to party with.