Yep, shoot toward your house.  When I lived in a subdivision I would shoot my bow from the back bedroom into the target that was on the couch. Cost me a couch and some drywall repair.  I don't blame your neighbor for calling the cops.