Abstract
In this article we study the fundamental solutions or ``$\alpha$-harmonic functions" for some nonlinear positive homogeneous nonlocal elliptic problems in conical domains, such as $$ \begin{eqnarray*}\label{ecbir1a1} {\mathcal F }(u)=0\ \ \hbox{in} \ \ \mathcal{C}_\omega,\quad u=0\ \ \hbox{in} \ \ \mathbb{R}^n\setminus \mathcal{C}_\omega ,\ \ \end{eqnarray*} $$ where $\omega$ is a proper $C^2$ domain in $S^{N-1}$ for $ N\geq 2$, $\mathcal{C}_\omega:=\{x\,|\,x\neq 0, {|x|^{-1}}x\in \omega\}$ is the cone-like domain related to $\omega$, and ${\mathcal F }$ is an extremal fully nonlinear integral operator. We prove the existence of two fundamental solutions that are homogeneous and do not change signs in the cone; one is bounded at the origin and the other at infinity. As an application, we use the fundamental solutions obtained to prove a Liouville type theorem in cones for supersolutions of Lane-Emden-Fowler equation of the form $$ \begin{eqnarray*}\label{eq 0.2} {\mathcal F }(u)+u^p = 0\ \ \hbox{in} \ \ \mathcal{C}_\omega, \quad u=0\ \ \hbox{in} \ \ \mathbb{R}^n\setminus \mathcal{C}_\omega. \end{eqnarray*} $$ We also prove a generalized Hopf type lemma in domains with corners. Most of our results are new even when ${\mathcal F }$ is the fractional Laplacian operator.

This publication has 32 references indexed in Scilit: