Maybe not a new OS, however it can be based on a minial unix/linux distribution plus the required amount of services setup on top.
Link - about this topic : https://www.informationweek.com/iot/8-iot-operating-systems-powering-the-future/d/d-id/1324464

IoT offering at present has more than 400 to 500 platforms and frameworks offered horizantal or vertical industry solutions.These are based on varied Linux kernels varied from memory foot print of few 100KBs to few 100MBs depennding on various factors such as hardware profile of the IoT and Auxilliary board, server/cloud data profile and many more design criterias.
During one of our design project, we had several iterations of Linux port to bring the kernel foor print to few 100KBs with streapping unwanted drivers such as network library, hard disk, display drivers etc. There are pros and cons of standardisation of OS.
Pros is good tested build for standardised kernel parameters but with IoT being large scope on I/O and data handling needed.
Cons:

1. Linux Kernel is sliding scale development thread so, getting into one version is challenging as most of the technology updates are always needed.
2. Various flavours of Linux - Ubantu, Kali, RedHat, Susse, Debian ( Most secured Linux version preferred by most of IoT designer) as well as FreeBSD as another OS also has similar issues.
3. Comptability with various hardware boards (Rasbery PI and others) and embeded CPU make ( Snapdragon, FreeScale, Cortex, NEC, Motorola,Intel and many more).

There is good framework available for Embedded Linux on Open standard which can be referred - http://www.openembedded.org
Long term, ASICs( Application Specific Integrate Circuits) with Linux/OS loaded on CPU can be manufactured which can cater to let us say various verticals - Home Automation, Industrial Automation - Retail, Defence Usage etc.

Which of the OS / kernels provide adquate security? Is that security sufficient?

Security is relative term as it's function of various aspects and forces not only at design and development stage but also at the implementation and usage stage. Each one has it's own use cases and security requirements to be trade in for cost, usability, time and complexity.
iOS is suppose to have record of more security control in terms of vulnerability at development and run time enviornment but license cost and other development cost are higher compared to other OS options so not widely preferred.
Same case is considered while developing using embedded Windows but atleast development and license cost are moderate so we see good push from Microsoft to put across it's usage on IoT platform as end to end offering. This does not obviate need for security control as hardening is still needed as well as lower foot print on memory.
There is one good aspect that is considered is of development and mainteance cost for security control. Even though there are in built library build by these OS vendors, still proper hardening is needed to bring more security control.
Android and Linux also have those native controls of security library as well as Debian kind of more secured version but, due to open system and code being available to intruders as well as developers, security control is always sliding scale code.
As I mentiond earlier, system design need to consider security as part of system design with criterias and considerations for development and run time enviornment. We have seen so many times during actual enterprise deployment IT enviornment detected that, such IoT or embedded system has several critical vulnerabilities.

IoT poses some particularly unique problems to get value out of the proliferation of devices and device types. Merely having access to a much broader and more diverse set of Internet-accessible devices isn't transformative, merely evolutionary.
The transformation occurs when a framework, fabric, yes, even "OS", for IoT allows analysis, deduction, machine learning, and prosaic business intelligence to be derived at the scale of IoT. Traditional solutions will not work due to the sheer volume of data, latency associated with collection, and the inherent delays from centrally processing terabytes to petabytes to exabytes of data. One startup I'm familiar with has a solution to this problem, where machine learning occurs at the Edge, allowing value to be extracted and used in a timely fashion. E.g. a faulty capacitor in a device is noted, a pattern emerges, and all devices can be remediated before costly failures, or, in the Google model, when a self-driving car makes a mistake, or learns from an event, every single other car now knows to not make that mistake. This is the holy grail of IoT, where understanding/knowledge is obtained and used quickly enough to be valuable.
Further, there are now management and monitoring frameworks that can realistically apply to IoT level device counts. Tanium is probably the best and most successful example of this.
There most assuredly will be at least one OS for IoT. Perhaps more than one to choose from over time. But, it won't resemble the typical central computing OS we've grown accustomed to in the past. Think RDD/Spark with simple-to-code plugins to draw inferences and "learn at the Edge/implement at the Edge", with a framework that efficiently distributes and shares data, all done in a secure fashion. It exists today in Alpha and Beta deliveries and soon will exist in the open market.

The future of IoT, in my view, is in the serverless event driven architecture. I beleive that reliance on any OS is no longer a relevant factor in multi (omni) platform ecosystems that are replacing the traditional single OS approach.
The real question is not so much the OS, but how Framework as a Service can fit into the IoT puzzle.
Virtualization and containerization are certainly one part of the solution. An OS, by definition, takes a monolithic approach. The future lays in an entirely new layer of abstraction that makes all underlying components work togehter seamlesly and securely to deliver microservices finely tuned to a particular end user need.
AWS and Azure already offer tools needed for building such solutions, including security considerations. Further development in this direction will make the user adoption easier and reduce relianceon writing custom code.